What if you set out to make the worst website you possibly could? So poorly designed and full of frustrating patterns that users would not only hate the experience of using this website, but would also come to hate your company. Could we make a web experience so terrible that it would express how much our company hated our users?
As a long-time Internet addict, I've encountered my fair share of terrible websites. Instagram where now half my feed is advertisements for stupid t-shirts and the other half is empty black space.
Who in the fuck would ever wear this
Or ARNGREN.net which is like if a newspaper ad threw up on my screen.
But Instagram still occasionally shows me pictures of people I follow and ultimately the stuff on ARNGREN is so cool I still want to buy it regardless of the layout.
No, I believe it is the crack team at Broadcom that have nailed it for the worst website in existence.
Lured in with free VMware
So through social media I discovered this blog post from VMware announcing that their popular virtualization software is free for personal use now. You can read that here. Great, I used VMware Fusion before and it was ok and maybe it will let me run Windows on an ARM Mac. Probably not but let's try it out and see.
This means that everyday users who want a virtual lab on their Mac, Windows or Linux computer can do so for free simply by registering and downloading the latest build from the new download portal located at support.broadcom.com. With the new commercial model, we have reduced our product group offerings down to a single SKU (VCF-DH-PRO) for users who require commercial use licensing. This simplification eliminates 40+ other SKUs and makes quoting and purchasing VMware Desktop Hypervisor apps, Fusion Pro and Workstation Pro, easier than ever. The new Desktop Hypervisor app subscription can be purchased from any Broadcom Advantage partner.
I don't want to register at support.broadcom.com but it looks like I don't have a choice as this is the screen on the VMware site.
Now this is where alarm bells start going crazy in my head. Nothing about this notice makes sense. "The store will be moving to a new domain". So it's...not...down for maintenance but actually is just gone? Or is it actually coming back? Because then you say "store will be shutdown" (just a quick note, you want "the store" and "will be shutting down on April 30th 2024"). Also why don't you just redirect to the new domain? What is happening here?
Broadcom
So then I go to support.broadcom.com which is where I was told to register and make an account.
First the sentence "Enhance your skills through multiple self-service avenues by creating your Broadcom Account" leaps off the page as just pure corporate nonsense. I've also never seen a less useful CAPTCHA, it looks like it is from 1998 and any modern text recognition software would defeat it. In fact the Mac text recognition in Preview defeats 3 of the 4 characters with no additional work:
So completely pointless and user hostile. Scoring lots of points for the worst website ever. I'm also going to give some additional points for "Ask our chatbot for assistance", an idea so revolting normally I'd just give up on the entire idea. But of course I'm curious, so I click on the link for the "Ask our chatbot" and.....
It takes me back to the main page.
Slow clap Broadcom. Imagine being a customer that is so frustrated with your support portal that you actually click "Ask a chatbot" and the web developers at Broadcom come by and karate chop you right in the throat. Bravo. Now in Broadcom's defense in the corner IS a chatbot icon so I kinda see what happened here. Let's ask it a question.
I didn't say hello. I don't know why it decided I said hello to it. But in response to VMware it gives me this:
Did the chatbot just tell me to go fuck myself? Why did you make a chatbot if all you do is select a word from a list and it returns the link to the support doc? Would I like to "Type a Query"?? WHAT IS A CHATBOT IF NOT TYPING QUERIES?
Next Steps
I fill in the AI-proof CAPTCHA and hit next, only to be greeted with the following screen for 30 seconds.
Finally I'm allowed to make my user account.
Um....alright....seems like overkill Broadcom but you know what this is your show. I have 1Password so this won't be a problem. It's not letting me copy/paste from 1Password into this field but if I do Command + \ it seems to let me insert. Then I get this.
What are you doing to me Broadcom. Did I....wrong you in some way? I don't understand what is happening. Ok well I refresh the page, try again and it works this time. Except I can't copy/paste into the Confirm Password field.
I mean they can't expect me to type out the impossibly complicated password they just had me generate right? Except they have and they've added a check to ensure that I don't disable Javascript and treat it like a normal HTML form.
Hey front-end folks, just a quick note. Never ever ever ever ever mess with my browser. It's not yours, it's mine. I'm letting you use it for free to render your bloated sites. Don't do this to me. I get to copy paste whatever I want whenever I want. When you get your own browser you can do whatever you want but while you are living in my house under my rules I get to copy/paste whenever I goddamn feel like it.
Quickly losing enthusiasm for the idea of VMware
So after pulling up the password and typing it in, I'm treated to this absolutely baffling screen.
Do I need those? I feel like I might need those. eStore at least sounds like something I might want. I don't really want Public Semiconductors Case Management but I guess that one comes in the box. 44 seconds of this icon later
I'm treated to the following.
Broadcom, you clever bastards. Just when I thought I was out, they pulled me back in. Tricking users into thinking a link is going to help them and then telling them to get fucked by advising them to contact your sales rep? Genius.
So then I hit cancel and get bounced back to......you guessed it!
Except I'm not even logged into my newly created account. So then I go to login with my new credentials and I finally make it to my customer portal. Well no first they need to redirect me back to the Broadcom Support main page again with new icons.
Apparently my name was too long to show and instead of fixing that or only showing first name Broadcom wanted to ensure the disrespect continued and sorta trail off. Whatever, I'm finally in the Matrix.
Now where might I go to...actually download some VMware software. There's a search bar that says "Search the entire site", let's start there!
Nothing found except for a CVE. Broadcom you are GOOD! For a second I thought you were gonna help me and like Lucy with the football you made me eat shit again.
My Downloads was also unhelpful.
But maybe I can add the entitlement to the account? Let's try All Products.
Of course the link doesn't work. What was I even thinking trying that? That one is really on me. However "All Products" on the left-hand side works and finally I find it. My white whale.
Except when I click on product details I'm brought back to....
The blank page with no information! Out of frustration I click on "My Downloads" again which is now magically full of links! Then I see it!
YES. Clicking on it I get my old buddy the Broadcom logo for a solid 2 minutes 14 seconds.
Now I have fiber internet with 1000 down, so this has nothing to do with me. Finally I click the download button and I get.....the Broadcom logo again.
30 seconds pass. 1 minute passes. 2 minutes pass. I'm not sure what to do.
No. No you piece of shit website. I've come too far and sacrificed too much of my human dignity. I am getting a fucking copy of VMware Fusion. Try 2 is the same thing. 3, 4, 5 all fail. Then finally.
I install it and like a good horror movie, I think it's all over. I've killed Jason. Except when I'm installing Windows I see this little link:
And think "wow I would like to know what the limitations are for Windows 11 for Arm!". Click on it and I'm redirected to...
Just one final fuck you from the team at Broadcom.
Conclusion
I've used lots of bad websites in my life. Hell, I've made a lot of bad websites in my life. But never before have I seen a website that so completely expresses just the pure hatred of users like this one. Everything was as poorly designed as possible, with user hostile design at every corner.
Honestly Broadcom, I don't even know why you bothered buying VMware. It's impossible for anyone to ever get this product from you. Instead of migrating from the VMware store to this disaster, maybe just shut this down entirely. Destroy the backups of this dumpster fire and start fresh. Maybe just consider a Shopify site because at least then an average user might have a snowballs chance in hell of ever finding something to download from you.
When I graduated college in 2008, even our commencement speaker talked about how moving back in with your parents is nothing to be ashamed of. I sat there thinking well that certainly can't be a good sign. Since I had no aspirations and my girlfriend was moving to Chicago, I figured why not follow her. I had been there a few times and there were no jobs in Michigan. We found a cheap apartment near her law school and I started job hunting.
After a few weeks applying to every job on Craigslist, I landed an odd job working for an Apple Authorized Repair Center. The store was in a strip mall in the suburbs of Chicago with a Dollar Store and a Chinese buffet next door. My primary qualifications were that I was willing to work for not a lot of money and I would buy my own tools. My interview was with a deeply Catholic boss who focused on how I had been an alter boy growing up. Like all of my bosses early on, his primary quality was he was a bad judge of character.
I was hired to do something that I haven't seen anyone else talk about on the Internet and wanted to record before it was lost to time. It was a weird program, a throwback to the pre-Apple Store days of Apple Mac support that was called AppleCare Dispatch. It still appears to exist (https://www.apple.com/support/products/mac/) but I don't know of any AASPs still dispatching employees. It's possible that Apple has subcontracted it out to someone else.
AppleCare Dispatch
Basically if you owned a desktop Mac and lived in certain geographic areas, when you contacted AppleCare to get warranty support they could send someone like me out with a part. Normally they'd do this only for customers who were extremely upset or had a store repair go poorly. I'd get a notice that AppleCare was dispatching a part, we'd get it from FedEx and then I'd fill a backpack full of tools and head out to you on foot.
While we had the same certifications as an Apple Genius, unlike the Genius Bar we weren't trained on any sort of "customer service" element. All we did was Mac hardware repairs all day, with pretty tight expectations of turnaround. So how it worked at the time was basically if the Apple Store was underwater with in-house repairs, or you asked for at-home or the customer was Very Important, we would get sent out. I would head out to you on foot with my CTA card.
That's correct, I didn't own a car. AppleCare didn't pay a lot for each dispatch and my salary of $25,000 a year plus some for each repair didn't go far in Chicago even in the Great Recession. So this job involved me basically taking every form of public transportation in Chicago to every corner of the city. I'd show up at your door within a 2 hour time window, take your desktop Mac apart in your house, swap the part, run the diagnostic and then take the old part with me and mail it back to Apple.
Apple provided a backend web panel which came with a chat client. Your personal Apple ID was linked with the web tool (I think it was called ASX) where you could order parts for repairs as well as open up a chat with the Apple rep there to escalate an issue or ask for additional assistance. The system worked pretty well, with Apple paying reduced rates for each additional part after the first part you ordered. This encouraged us all to get pretty good at specific diagnostics with a minimal number of swaps.
Our relationship to Apple was bizarre. Very few people at Apple even knew the program existed, seemingly only senior AppleCare support people. We could get audited for repair quality, but I don't remember that ever happening. Customer satisfaction was extremely important and basically determined the rate we got paid, so we were almost never late to appointments and typically tried to make the experience as nice as possible. Even Apple Store staff seemed baffled by us on the rare occasions we ran into each other.
There weren't a lot of us working in Chicago around 2008-2010, maybe 20 in total. The community was small and I quickly met most of my peers who worked at other independent retail shops. If our customer satisfaction numbers were high, Apple never really bothered us. They'd provide all the internal PDF repair guides, internal diagnostic tools and that was it.
It is still surprising that Apple turned us loose onto strangers without anyone from Apple speaking to us or making us watch a video. Our exam was mostly about not ordering too many parts and ensuring we could read the PDF guide of how to fix a Mac. A lot of the program was a clear holdover from the pre-iPod Apple, where resources were scarce and oversight minimal. As Apple Retail grew, the relationship to Apple Authorized Service Providers got more adversarial and controlling. But that's a story for another time.
Tools etc
For the first two years I used a Manhattan Portage bag, which looked nice but was honestly a mistake. My shoulder ended up pretty hurt after carrying a heavy messenger bag for 6+ hours a day.
The only screwdrivers I bothered with was Wiha precision screwdrivers. I tried all of them and Wiha were consistently the best by a mile. Wiha has a list of screwdrivers by Apple model available here: https://www.wihatools.com/blogs/articles/apple-and-wiha-tools
Macs of this period booted off of FireWire, so that's what I had with me. FireWire 800 LaCie drives were the standard issue drives in the field.
You'd partition it to have a series of OS X Installers on there (so you could restore the customer back to what they had before) along with a few bootable installs of OS X. These were where you'd run your diagnostic software. The most commonly used ones were as follows:
Remember back when Macs were something you could fix? Crazy times
9/11 Truther
One of my first calls was for a Mac Pro at a private residence. It was a logic board, which means the motherboard of the Mac. I wasn't thrilled, because removing and replacing the Mac Pro logic board was a time-consuming repair that required a lot of light. Instead of a clean workspace with bright lights I got a guy who would not let me go until I had watched how 9/11 was an inside job.
The logic board in question
"Look, you don't really think the towers were blown up by planes do you?" he said as he dug around this giant stack of papers to find...presumably some sort of Apple-related document. I had told him that I had everything I needed, but that I had a tight deadline and needed to start right now. "Sure, but I'll put the video on in the background and you can just listen to it while you work." So while I took a Mac Pro down to the studs and rebuilt it, this poorly narrated video explained how it was the CIA behind 9/11.
His office or, "command center", looked like a set of the X-Files. There were folders and scraps of paper everywhere along with photos of buildings, planes, random men wearing sunglasses. I think it was supposed to come across as if he was doing an investigation, but it reminded me more of a neighbor who struggled with hoarding. If there was an organizational system, I couldn't figure it out. Why was this person so willing to dedicate an large portion of their house to "solving a mystery" the rest of us had long since moved on from?
The Mac Pro answered all my questions when it booted up. The desktop was full of videos he had edited of 9/11 truth material along with website assets for where he sold these videos. This guy wasn't just a believer, he produced the stuff. When I finished, we had to run a diagnostic test to basically confirm the thing still worked as well as move the serial number onto the logic board. When it cleared diagnostic I took off, thanking him for his time and wishing him a nice day. He looked devastated and asked if I wanted to go grab a drink at the bar and continue our conversation. I declined, jogging to the L.
The Doctors
One of the rich folks I was sent out to lived in one of those short, super expensive buildings on Lake Shore Drive. For those unfamiliar, these shorter buildings facing the water in Chicago are often divided into a few large houses. Basically you pass through an army of doorman, get shown into an elevator that opens into the persons house. That was, if you could get through the doormen.
The staff in rich peoples houses want to immediately establish with any contractor coming into the home that they're superior to you. This happened to me constantly, from personal assistants to doormen, maids, nannies, etc. Doormen in particular liked to make a big deal of demonstrating that they could stop me from going up. This one stuck out because he made me take the freight elevator, letting me know "the main elevator is for people who live here and people who work here". I muttered about how I was also working there and he rolled his eyes and called me an asshole.
On another visit to a different building I had a doorman physically threaten "to throw me down" if I tried to get on the elevator. The reason was all contractors had to have insurance registered with the building before they did work there, even though I wasn't.....removing wires from the wall. The owner came down and explained that I wasn't going to do any work, I was just "a friend visiting". I felt bad for the doorman in that moment, in a dumb hat and ill-fitting jacket with his brittle authority shattered.
So I took the freight elevator up, getting let into what I would come to see as "the rich persons template home". My time going into rich peoples houses were always disappointing, as they are often a collection of nice items sort of strewn around. I was shown by the husband into the library, a beautiful room full of books with what I (assumed) were prints of paintings in nice frames leaning against the bookshelves. There was an iMac with a dead hard drive, which is an easy repair.
The process for fixing a hard drive was "boot to DiskWarrior, attempt to fix disk, have it fail, swap the drive". Even if DiskWarrior fixed the Mac and it booted, I would still swap the drive (why not and it's what I was paid to do) but then I didn't have to have the talk. This is where I would need to basically sit someone down and tell them their data was gone. "What about my taxes?!?" I would shake my head sadly. Thankfully this time the drive was still functional so I could copy the data over with a SATA to USB adapter.
As I reinstalled OS X, I walked around the room and looked at the books. I realized they were old, really old and the paintings on the floor were not prints. There were sketches by Picasso, other names I had heard in passing through going through art museums. When he came back in, I asked why there was a lot of art. "Oh sure, my dads, his big collection, I'm going to hang it up once we get settled." He, like his wife, didn't really acknowledge my presence unless I directed a question right at him. I started to google some of the books, my eyes getting wide. There was millions of dollars in this room gathering dust. He never made eye contact with me during this period and quickly left the room.
This seems strange but was really common among these clients. I truly think for many of the C-level type people whose house I went to, they didn't really even see me. I had people turn the lights off in rooms I was in, forget I was there and leave (while arming the security system). For whatever reason I instantly became part of the furniture. When I went to the kitchen for a drink of water, the maid let me know that they have lived there for coming up on 5 years.
This was surprising to me because the apartment looked like they had moved in two weeks ago. There were still boxes on the floor, a tv sitting on the windowsill and what I would come to understand was a "prop fridge". It had bottled water, a single bottle of expensive champagne, juices, fruit and often some sort of energy drink. No leftovers, everything gets swapped out before it goes bad and gets replaced. "They're always at work" she explained, grabbing her bag and offering to let me out before she locked up. They were both specialist doctors and this was apparently where they recharged their batteries.
After the first AppleCare Dispatch visit they would call me back for years to fix random problems. I don't think either of them ever learned my name.
HARPO Studio
I was once called to fix a "high profile" computer at HARPO studios in Chicago. This was where they filmed the Oprah Winfrey Show, which I obviously knew of the existence of but had never watched. Often these celebrity calls went to me, likely because I didn't care and didn't particularly want them. I was directed to park across the street and told even though the signs said "no parking" that they had a "deal with the city".
This repair was suspicious and I got the sense that someone had name dropped Oprah to maybe get it done. AppleCare rarely sent me multiple parts unless the case was unusual or the person had gotten escalated through the system. If you emailed Steve Jobs back in the day and his staff approved a repair, it attached a special code to the serial number that allowed us to order effectively unlimited items against the serial number. However with the rare "celebrity" case, we would often find AppleCare did the same thing, throwing parts at us to make the problem go away.
The back area of HARPO was busy, with what seemed like an almost exclusively female staff. "Look it's important that if you see Oprah, you act normally, please don't ask her for an autograph or a photo". I nodded, only somewhat paying attention because never in a million years would I do that. This office felt like the set of The West Wing, with people constantly walking and talking along with a lot of hand motions. My guide led me to a back office with a desk on one side and a long table full of papers and folders. The woman told me to "fix the iMac" and left the room.
Not the exact office but you get the jist
I swapped the iMac hard drive and screen, along with the memory and wifi then dived under the desk the second Oprah walked in. The woman and Oprah had a conversation about scheduling someone at a farm, or how shooting at a farm was going and then she was gone. When I popped my head up, the woman looked at me and was like "can you believe you got to meet Oprah?" She had a big smile, like she had given me the chance of a lifetime.
The bummer about the aluminum iMac repairs is you have to take the entire screen off to get anything done. This meant I couldn't just run away and hide my shame after effectively diving under a table to escape Oprah, a woman who I am certain couldn't have cared less what came out of my mouth. I could have said "I love to eat cheese sometimes" and she would have nodded and left the room.
So you have to pop the glass off (with suction cups, not your hands like a psycho as shown above), then unscrew and remove the LCD and then finally you get access to the actual components. Any dust that got on the LCD would stick and annoy people, so you had to try and keep it as clean as possible while moving quickly to get the swap done. The nightmare was breaking the thick cables that connected the screen to the logic board, something I did once and required a late night trip to an electronics repair guy who got me sorted out with a soldering iron.
The back alley electronics repair guy is the dark secret of the Dispatch world. If you messed up a part, pulled a cable or broke a connector, Apple could ask you to pay for that part. The Apple list price for parts were hilariously overpriced. Logic boards were like $700-$900, each stick of RAM was like $90 for ones you could buy on crucial for $25. This could destroy your pay for that month, so you'd end up going to Al, who ran basically a "solder Apple stuff back together" business in his garage. He wore overalls and talked a lot about old airplanes, which you'd need to endure in order to get the part fixed. Then I'd try to get the part swapped and just pray that the thing would turn on long enough for you to get off the property. Ironically his parts often lasted longer than the official Apple refurbished parts.
After I hid under the desk deliberately, I lied for years afterwards, telling people I didn't have time to say hi. In reality my mind completely blanked when she walked in. I stayed under the desk because I was nervous that everyone was going to look at me to be like "I loved when you did X" and my brain couldn't form a single memory of anything Oprah had ever done. I remembered Tom Cruise jumping on a couch but I couldn't recall if this was a good thing or a bad thing when it happened.
Oh and the car that I parked in the area the city didn't enforce? It had a parking ticket, which was great because I had borrowed the car. Most of the payment from my brush with celebrity went to the ticket and a tank of gas.
Brownstone Moms
One of the most common calls I got was to rich peoples houses in Lincoln Park, Streeterville, Old Town and a few other wealthy neighborhoods. They often live in distinct brownstone houses with small yards with a "public" entrance in the front, a family entrance on the side and then a staff entrance through the back or in the basement.
These houses were owned by some of the richest people in Chicago. The houses themselves were beautiful, but they don't operate like normal houses. Mostly they were run by the wives, who often had their own personal assistants. It was an endless sea of contractors coming in and out, coordinated by the mom and sometimes the nanny.
Once I was there, they'd pay me to do whatever random technical tasks existed outside of the initial repair. I typically didn't mind since I was pretty fast at the initial repair and the other stuff was easy, mostly setting up printers or routers. The sense I got was if the household made the AppleCare folks life a living hell, I would get sent out to make the problem disappear. These people often had extremely high expectations of customer service, which could be difficult at times.
There was a whole ecosystem of these small companies I started to run into more and more. They seemed to specialize in catering to rich people, providing tutoring services, in-house chefs, drivers, security and every other service under the sun. One of the AV installation companies and I worked together off the books after-hours to set up Apple TVs and Mac Minis as the digital media hubs in a lot of these houses. They'd pay me to set up 200 iPods as party favors or wire an iPad into every room.
Often I'd show up only to tell them their hard drive was dead and everything was gone. This was just how things worked before iCloud Photos, nobody kept backups and everything was constantly lost forever. Here they would often threaten or plead with me, sometimes insinuating they "knew people" at Apple or could get me fired. Jokes on you people, I don't even know people at Apple was often what ran through my head. Threats quickly lost their power when you realized nobody at any point had asked your name or any information about yourself. It's hard to threaten an anonymous person.
The golden rule that every single one of these assistants warned me about was not to bother the husband when he gets home. Typically these CEO-types would come in, say a few words to their kids and then retreat to their own area of the house. These were often TV rooms or home theaters, elaborate set pieces with $100,000+ of AV equipment in there that was treated like it was a secret lair of the house. To be clear, none of these men ever cared at all that I was there. They didn't seem to care that anybody was there, often barely acknowledging their wives even though an immense amount of work had gone into preparing for his return.
As smartphones became more of a thing, the number of "please spy on my teen" requests exploded. These varied from installing basically spyware on their kids laptops to attempting to install early MDM software on the kids iPhones. I was always uncomfortable with these jobs, in large part because the teens were extremely mean to me. One girl waited until her mom left the room to casually turn to me and say "I will pay you $500 to lie to my mom and say you set this up".
I was offended that this 15 year old thought she could buy me, in large part because she was correct. I took the $500 and told the mom the tracking software was all set up. She nodded and told me she would check that it was working and "call me back if it wasn't". I knew she was never going to check, so that part didn't spook me. I just hoped the kid didn't get kidnapped or something and I would end up on the evening news. But I was also a little short that month for rent so what can you do.
Tip for anyone reading this looking to get into this rich person Mac business
So the short answer is Time Machine is how you get paid month after month. Nobody checks Time Machine or pays attention to the "days since" notification. I wrote an AppleScript back in the day to alert you to Time Machine failures through email, but there is an app now that does the same thing: https://tmnotifier.com/
Basically when the backups fail, you schedule a visit and fix the problem. When they start to run out of space, you buy a new bigger drive. Then you backup the Time Machine to some sort of encrypted external location so when the drive (inevitably) gets stolen you can restore the files. The reason they keep paying you is you'll get a call at some point to come to the house at a weird hour and recover a PDF or a school assignment. That one call is how you get permanent standing appointments.
Nobody will ever ask you how it works, so just find the system you like best and do that. I preferred local Time Machine over something like remote backup only because you'll be sitting there until the entire restore is done and nothing beats local. Executives will often fill the "family computer" with secret corporate documents they needed printed off, so be careful with these backups. Encrypt, encrypt, encrypt then encrypt again. Don't bother explaining how the encryption works, just design the system with the assumption that someone will at some point put a CSV with your social security number onto this fun family iMac covered in dinosaur stickers.
Robbed for Broken Parts
A common customer for repairs would be schools, who would work with Apple to open a case for 100 laptops or 20 iMacs at a time. I liked these "mass repair" days, typically because the IT department for Chicago Public Schools would set us up with a nice clean area to work and I could just listen to a podcast and swap hard drives or replace top cases. However this mass repair was in one of Chicago's rougher neighborhoods.
Personal safety was a common topic among the dispatch folks when we would get together for a pizza and a beer. Everyone had bizarre stories but I was the only one not working out of my car. The general sense among the community was it was not an "if" but a "when" until you were robbed. Typically my rule was if I started to get nervous I'd "call back to the office" to check if a part had arrived. Often this would calm people down, reminding them that people knew where I was. Everyone had a story of getting followed back to their car and I had been followed back to the train once or twice.
On this trip though everything went wrong that could go wrong. My phone, the HTC Dream running v1 of Android had decided to effectively "stop phoning". It was still on but decided we were not, in fact, in the middle of a large city. I was instead in a remote forest miles away from a cell tower. I got to the school later than I wanted to be there, showing up at noon. When I tried to push it and come back the next day the staff let me know the janitors knew I would be there and would let me out.
So after replacing a ton of various Mac parts I walked out with boxes of broken parts in my bag and a bunch in an iMac box that someone had given me. My plan was I would head back home, get them checked in and labeled and then drop them off at a FedEx store. When I got out and realized it was dark, I started to accept something bad was likely about to happen to me. Live in a city for any amount of time and you'll start to develop a subconscious odds calculator. The closing line on this wasn't looking great.
Sure enough while waiting for the bus, I was approached by a man who made it clear he wanted the boxes. He didn't have a weapon but started to go on about "kicking the shit" out of me and I figured that was good enough for me. He clearly thought there was an iMac in the box and I didn't want to be here when he realized that wasn't true. I handed over my big pile of broken parts and sprinted to the bus that was just pulling up, begging the driver to keep driving. As a CTA bus driver, he had of course witnessed every possible horror a human can inflict on another human and was entirely unphased by my outburst. "Sit down or get off the bus".
When I got home I opened a chat with as Apple rep who seemed unsure of what to do. I asked if they wanted me to go to the police and the rep said if I wanted to I could, but after "talking to some people on this side" they would just mark the parts as lost in transit and it wouldn't knock my metrics. I thanked them and didn't think much more of the incident until weeks later when someone from Apple mailed me a small Apple notebook.
They never directly addressed the incident (truly the notebook might be unrelated) but I always thought the timing was funny. Get robbed, get a notebook. I still have the notebook.
A few years ago I became aware of the existence of the Apple Pro Workflow teams. These teams exist inside Apple to provide feedback to the owners of various professional-geared hardware and software teams inside the company. This can be everything from advising the Mac Pro team what kind of expansion a pro workstation might need all the way to feedback to the Logic Pro and Final Cut teams on ways the make the software fit better into conventional creative workflows.
I think the idea is really clever and wish more companies did something like this. It would be amazing, for instance, if I worked on accounting software if we had a few accountants in-house attempting to just use our software to solve problems. Shortening the feedback loop and relying less on customers reporting problems and concerns is a great way of demonstrating to people you take their time seriously. However it did get me thinking: what would a Developer Pro Workflow team ask for?
The History of the MacBook Pro and me
I've owned MacBook Pros since they launched, with the exception of the disastrous TouchBar Mac with the keyboard that didn't work. While I use Linux every day for my work, I prefer to have macOS as a "base operating system". There are a lot of reasons for that, but I think you can break them down into a few big advantages to me:
The operating system is very polished. "Well wait, maybe you haven't tried Elementary/Manjaro/etc." I have and they're great for mostly volunteer efforts, but it's very hard to beat the overall sense of quality and "pieces fitting together" that comes with macOS. This is an OS maintained by , I suspect, a large team in terms of software development team sizes. Improvements and optimizations are pretty common and there is a robust community around security. It doesn't just work but for the most part it does keep ticking along.
The hardware is able to be fixed around the world. One thing that bothered me about my switch to Linux: how do I deal with repairs and replacements. PCs change models all the time and, assuming I had a functional warranty, how long could I go without a computer? I earn my living on this machine, I can't wait six weeks for a charger or spend a lot of time fighting with some repair shop on what needs to be done to fix my computer. Worst case I can order the exact same model I have now from Apple, which is huge.
The third-party ecosystem is robust, healthy and frankly time-tested. If I need to join a Zoom call, I don't want to think about whether Zoom is going to work. If someone calls me on Slack, I don't want to deal with quitting and opening it four times to get sound AND video to work. When I need to use third-party commercial software its often non-optional (a customer requires that I use it) and I don't have a ton of time to debug it. With the Mac, commercial apps are just higher quality than Linux. They get a lot more attention internally from companies and they just have a much higher success rate of working.
Suggestions from my fake Workflow team
Now that we've established why I like the platform, what could be done to improve it for developers? How could we take this good starting point and further refine it. These suggestions are not ranked.
An official way to run Linux
Microsoft changed the conversation in developer circles with Windows Subsystem for Linux. Suddenly Windows machines, previously considered inferior for developing applications for Linux, mostly had that limitation removed. The combination of that and the rise of containerization has really reduced the appeal of Macs for developers, as frankly you just don't use the base OS UNIX tooling as much as you used to.
If Apple wants to be serious about winning back developer mindshare, they wouldn't need to make their own distro. I think something like Canonical Multipass would work great, still giving us a machine that resembles the cloud computers we're likely going to be deploying to. Making them more of a first-class citizen inside the Apple software stack like Microsoft would be a big win.
Well why would Apple do this if there are already third-party tools? Part of it is marketing, Apple just can tell a lot more users about something like this. Part of it is that it signals a seriousness about the effort and a commitment on their part to keep it working. I would be hesitant to rely heavily on third-party tooling around the Apple hyperkit VM system just because I know at any point without warning Apple could release new hardware that doesn't work with my workflow. If Apple took the tool in-house there would be at least some assurances that it would continue to work.
Podman for the Mac
Docker Desktop is no longer a product you should be using. With the changes to their licensing, Docker has decided to take a much more aggressive approach to monetization. It's their choice, obviously, but to me it invalidates using Docker Desktop for even medium sized companies. The license terms of: fewer than 250 employees AND less than $10 million in annual revenue means I can accidentally violate the license by having a really good year, or merging with someone. I don't need that stress in my life and would never accept that kind of aggressive license in business-critical software unless there was absolutely no choice.
Apple could do a lot to help with me by assisting with the porting of podman to the Mac. Container-based workflows aren't going anywhere and if Apple installed podman as part of the "Container Developer Tools" command, it would not only remove a lot of concerns from users about Docker licensing, but also would just be very nice. Again this is solvable by users using a Linux VM but its a clunky solution and nothing something I think of as very Apple-like. If there is a team sitting around thinking about the button placement in Final Cut Pro, making my life easier when running podman run for the 12th time would be nice as well.
Xcode is the worst IDE I've ever used and needs a total rewrite
I don't know how to put this better. I'm sure the Xcode team is full of nice people and I bet they work hard. It's the worst IDE I've ever used. Every single thing in Xcode is too slow. I'm talking storyboards, suggestions, every single thing drags. A new single view app, just starting from scratch, takes multiple minutes to generate.
You'll see this screen so often, you begin to wonder where "Report" even goes
Basic shit in Xcode doesn't work. Interface Builder is useless, autocomplete is random in terms of how well or poorly it will do that second, even git commands don't work all the time for me inside Xcode. I've never experienced anything like it with free IDEs, so the idea that Apple ships this software for actual people to use is shocking to me.
If you are a software developer who has never tried to make a basic app in Xcode and are curious what people are talking about, give it a try. I knew mobile development was bad, but spending a week working inside Xcode after years of PyCharm and Vim/Tmux, I got shivers imagining if I paid my mortage with this horrible tool. Every 40 GB update you must just sweat bullets, worried that this will be the one that stops letting you update your apps.
I also cannot imagine that people inside Apple use this crap. Surely there's some sort of special Xcode build that's better or something, right? I would bet money a lot of internal teams at Apple are using AppCode to do their jobs, leaving Xcode hidden away. But I digress: Xcode is terrible and needs a total rewrite or just call JetBrains and ask how much they would charge to license every single Apple Developer account with AppCode, then move on with your life. Release a smaller standalone utility just for pushing apps to the App Store and move on.
It is embarrassing that Apple asks people to use this thing. It's hostile to developers and it's been too many years and too many billions of dollars made off the back of App developers at this point. Android Studio started out bad and has gotten ok. In that time period Xcode started pretty bad and remains pretty bad. Either fix it or give up on it.
Make QuickLook aware of the existence of code
QuickLook, the Mac functionality where you click on a file and hit spacebar to get a quick look at it is a great tool. You can quickly go through a ton of files and take a quick glance at all of them, or at least creative professionals can. I want to as well. There's no reason macOS can't understand what yaml is, or how to show me JSON in a non-terrible way. There are third party tools that do this but I don't see this as something Apple couldn't bring in-house. It's a relatively low commitment and would be just another nice daily improvement to my workflow.
Look how great that is!
An Apple package manager
I like Homebrew and I have nothing but deep respect for the folks who maintain it. But it's gone on too long now. The Mac App Store is a failure, the best apps don't live there and the restrictions and sandboxing along with Apple's cut means nobody is rushing to get listed there. Not all ideas work and its time to give up on that one.
Instead just give us an actual package manager. It doesn't need to be insanely complicated, heck we can make it similar to how Apple managed their official podcast feed for years with a very hands-off approach. Submit your package along with a URL and Apple will allow users a simple CLI to install it along with declared dependencies. We don't need to start from scratch on this, you can take the great work from homebrew and add some additional validation/hosting.
How great would it be if I could write a "setup script" for new developers when I hand them a MacBook Pro that went out and got everything they needed, official Apple apps, third-party stuff, etc? You wouldn't need some giant complicated MDM solution or an internal software portal and you would be able to add some structure and vetting to the whole thing.
Just being able to share a "new laptop setup" bash script with the internet would be huge. We live in a world where more and more corporations don't take backups of work laptops and they use tooling to block employees from maintaining things like Time Machine backups. While it would be optimal to restore my new laptop from my old laptop, sometimes it isn't possible. Or maybe you just want to get rid of the crap built up over years of downloading stuff, opening it once and never touching it again.
To me this is a no-brainer, taking the amazing work from the homebrew folks, bringing them in-house and adding some real financial resources to it, with the goal of making a robust CLI-based package installation process for the Mac. Right now Homebrew developers actively have to work around Apple's restrictions to get their tool to work, a tool tons of people rely on every day. That's just not acceptable. Pay the homebrew maintainers a salary, give it a new Apple name and roll it out at WWDC. Everybody will love you and you can count the downloads as "Mac App Store" downloads during investor calls.
A git-aware text editor
I love BBedit so much I bought a sweatshirt for it and a pin I proudly display on my backpack. That's a lot of passion for a text editor, but BBedit might be the best software in the world. I know, you are actively getting upset with me as you read that, but hear me out. It just works, it never loses files and it saves you so much time. Search and replace functionality in BBedit has, multiple times, gotten me out of serious jams.
But for your average developer who isn't completely in love with BBedit, there are still huge advantages to Apple shipping even a much more stripped down text editor that Markdown and Git-functionality. While you likely have your primary work editor, be it vim for me or vscode for you, there are times when you need to make slight tweaks to a file or you just aren't working on something that needs the full setup. A stripped version of BBedit or something similar that would allow folks to write quick docs, small shell scripts or other tasks that you do a thousand times a year would be great.
This doesn't have to be the greatest text editor ever made, but it would be a huge step forward for the developer community to have something that works out of the box. Plus formats like Markdown are becoming more common in non-technical environments.
Why would Apple do this if third-party apps exist?
For the same reason they make iPhoto even though photo editors exist. Your new laptop from Apple should be functional out of the box for a wide range of tasks and adding a text editor that can work on files that lots of Apple professional users interact with on a daily basis underscores how seriously Apple takes that market. Plus maybe it'll form the core of a new Xcode lite, codenamed Xcode functional.
An Apple monitor designed for text
This is a bit more of a reach, I know that. But Apple could really use a less-expensive entry into the market and in the same way the Pro Display XDR is designed for the top-end of the video editing community, I would love something like that but designed more for viewing text. It wouldn't have nearly the same level of complexity but it would be nice to be able to get a "full end to end" developer solution from Apple again.
IPS panels would be great for this, as the color accuracy and great viewing angels would be ideal for development but you won't care about them being limited to 60hz. A 27 inch panel is totally sufficient and I'd love to be able to order them at the same time as my laptop for a new developer. There are lots of third-party alternatives but frankly dealing with the endless word soup that is monitors these days is difficult to even begin to track. I love my Dell UltraSharp U2515H but I don't even know how long I can keep buying them or how to find the successor to it when it gets decommissioned.
Actually I guess its already been decommissioned and also no monitors are for sale.
What did I miss?
What would you add to the MacBook Pro to make it a better developer machine? Let me know at @duggan_mathew on twitter.
Private Relay is an attempt by Apple to change the way traffic is routed from user to internet service and back. This is designed to break the relationship between user IP address and information about that user, reducing the digital footprint of that user and eliminating certain venues of advertising information.
It is a new feature in the latest version of iOS and MacOS that will be launching in "beta mode". It is available to all users who pay Apple for iCloud storage and I became interested in it after watching the WWDC session about preparing for it.
TL;DR
Private Relay provides real value to users, but also fundamentally changes the way network traffic flows across the internet for those users. Network administrators, programmers and owners of businesses which rely on IP addresses from clients for things like whitelisting, advertising and traffic analysis should be aware of this massive change. It is my belief that this change is not getting enough attention in the light of the CSAM scanning.
What happens when you turn on Private Relay?
The following traffic is impacted by Private Relay
All Safari web browsing
All DNS queries
All insecure HTTP traffic
Traffic from those sources will no longer take the normal route to their destination, instead being run through servers controlled by either Apple or its partners. They will ingress at a location close to you and then egress somewhere else, with an IP address known to be from your "region". In theory websites will still know roughly where you are coming from, but won't be able to easily combine that with other information they know about your IP address to enrich targeted advertisements. Access logs and other raw sources of data will also be less detailed, with the personally identifiable information that is your IP address no longer listed on logs for every website you visit.
Why is Apple doing this?
When you go to a website, you are identified in one of a thousand ways, from cookies to device fingerprinting. However one of the easiest ways is through your IP address. Normal consumers don't have "one" IP address, they are either given one by their ISP when their modem comes online and asks for one, or their ISP has them behind "carrier-grade NAT". So normally what happens is that you get your modem, plug it in, it receives an IP address from the ISP and that IP addresses identifies you to the world.
Normally how the process works is something like this:
Your modems MAC address appears on the ISPs network and requests an IP address
The ISP does a lookup for the MAC address, makes sure it is in the table and then assigns an IP, ideally the same IP over and over again so whatever cached routes from the ISPs side exist are still used.
All requests from your home are mapped to a specific IP addresses and, over time, given the combination of other information about the browsing history and advertising data, it is possible to combine the data together to know where you live and who you are within a specific range.
You can see how close the geographic data is by checking out the map available here. For me it got me within a few blocks of my house, which is spooky.
CGNAT
Because of IPv4 address exhaustion, it's not always possible to assign every customer their own IP address. You know you have a setup like this because the IP address your router gets is in the "private range" of IP addresses, but when you go to IP Chicken you'll have a non-private IP address.
For those interested you can get more information about how CGNAT works here.
Doesn't my home router do that?
Yeah so your home router kind of does something similar with its own IP address ranges. So next time a device warns you about "double-NAT" this might be what it is talking about, basically nested NAT. (Most often double-NAT is caused by your modem also doing NAT though.) Your home router runs something called PAT or PAT in overload. I think more often it is called NAPT in modern texts.
This process is not that different from what we see above. One public IP address is shared and the different internal targets are identified with ports. Your machine makes an outbound connection, your router receives the request and rewrites the packet with a random high port. Every outbound connection gets its own entry in this table.
So during the normal course of using the internet, your IP address is exposed to the following groups
Every website or web service you are connecting to
Your DNS server also can have a record of every website you looked up.
Your ISP can obviously see where every request to and from your home went to
This means there are three groups of people able to turn your request into extremely targeted advertising. The most common one I see using IP address is hyper-local advertising. If you have ever gotten an online ad for a local business or service and wondered "how did they know it was me", there is a good chance it was through your IP.
DNS is one I think is often forgotten in the conversation about leaking IPs, but since it is a reasonable assumption that if you make a DNS lookup for a destination you will go to that destination, it is as valuable as the more invasive systems without requiring nearly as much work. Let's look at one popular example, Google DNS.
Google DNS
Turkish protests relies on twitter and used Google DNS once it was blocked
The famous 8.8.8.8. Google DNS has become famous because of the use of DNS around the world as a cheap and fast way to block network access for whole countries or regions. A DNS lookup is just what turns domain names into IP address. So for this site:
➜ ~ host matduggan.com
matduggan.com has address 67.205.139.103
Since DNS servers are normally controlled by ISPs and subject to local law, it is trivial if your countries leadership wants to block access for users to get to Twitter by simply blocking lookups to twitter.com. DNS is a powerful service that is normally treated as an afterthought. Alternatives came up, the most popular being Google DNS. But is it actually more secure?
Google asserts that they only store your IP address for 24-48 hours in their temporary logs. When they migrate your data to their permanent DNS logs, they remove IP address and replace with region data. So instead of being able to drill down to your specific house, they will only be able to tell your city. You can find more information here. I consider their explanation logical though and think they are certainly more secure when compared to a normal ISP DNS server.
Most ISPs don't offer that luxury, simply prefilling their DNS servers when you get your equipment from them and add it to the network. There is very little information about what they are doing with it that I was able to find, but they are allowed now to sell that information if they so choose. This means the default setting for US users is to provide an easy to query copy of every website their household visits to their ISP.
So most users will not take the proactive step to switch their DNS servers to one provided by Google or other parties. However since most folks won't do that, the information is just being openly shared with whoever has access to that DNS server.
NOTE: If you are looking to switch your DNS servers off your ISP, I recommend dns.watch. I've been using them for years and feel strongly they provide an excellent service with a minimum amount of fuss.
How does Private Relay address these concerns?
DNS
This is how a normal DNS lookup works.
Apple and Cloudflare engineers have proposed a new standard, which they discuss in their blog post here. ODNS or "oblivious DNS" is a system which allows clients to mask the originator of the request from the server making the lookup, breaking the IP chain.
This is why all DNS queries are getting funneled through Private Relay, removing the possibility of ISP DNS servers getting this valuable information. It is unclear to me in my testing if I am using Apple's servers or Cloudflares 1.1.1.1 DNS service. With this system it shouldn't matter in terms of privacy.
2. Website IP Tracking
When on Private Relay, all traffic is funneled first through an Apple ingress service and then out through a CDN partner. Your client makes a lookup to one of these two DNS entries using our new fancy ODNS:
mask.icloud.com
mask-h2.icloud.com
This returns a long list of IP addresses for you to choose from:
mask.icloud.com is an alias for mask.apple-dns.net.
mask.apple-dns.net has address 172.224.41.7
mask.apple-dns.net has address 172.224.41.4
mask.apple-dns.net has address 172.224.42.5
mask.apple-dns.net has address 172.224.42.4
mask.apple-dns.net has address 172.224.42.9
mask.apple-dns.net has address 172.224.41.9
mask.apple-dns.net has address 172.224.42.7
mask.apple-dns.net has address 172.224.41.6
mask.apple-dns.net has IPv6 address 2a02:26f7:34:0:ace0:2909::
mask.apple-dns.net has IPv6 address 2a02:26f7:36:0:ace0:2a05::
mask.apple-dns.net has IPv6 address 2a02:26f7:36:0:ace0:2a07::
mask.apple-dns.net has IPv6 address 2a02:26f7:34:0:ace0:2904::
mask.apple-dns.net has IPv6 address 2a02:26f7:34:0:ace0:2905::
mask.apple-dns.net has IPv6 address 2a02:26f7:36:0:ace0:2a04::
mask.apple-dns.net has IPv6 address 2a02:26f7:36:0:ace0:2a08::
mask.apple-dns.net has IPv6 address 2a02:26f7:34:0:ace0:2907::
These IP addresses are owned by Akamai and are here in Denmark, meaning all Private Relay traffic first goes to a CDN endpoint. These are globally situated datacenters which allow companies to cache content close to users to improve response time and decrease load on their own servers. So then my client opens a connection to one of these endpoints using a new protocol, QUIC. Quick, get it? Aren't network engineers fun.
QUIC integrates TLS to encrypt all payload data and most control information. Its based on UDP for speed but is designed to replace TCP, the venerable protocol that requires a lot of overhead in terms of connections. By baking in encryption, Apple is ensuring a very high level of security for this traffic with a minimum amount of trust required between the partners. It also removes the loss recovery elements of TCP, instead shifting that responsibility to each QUIC stream. There are other advantages such as better shifting between different network providers as well.
So each user makes an insecure DNS lookup to mask.apple-dns.net, establishes a QUIC connection to the local ingress node and then that traffic is passed through to the egress CDN node. Apple maintains a list of those egress CDN nodes you can see here. However users can choose whether they want to reveal even city-level information to websites through the Private Relay settings panel.
If I choose to leave "Maintain General Location" checked, websites will know I'm coming from Copenhagen. If I select the "Country and Time Zone" you just know I'm coming fron Denmark. The traffic will appear to be coming from a variety of CDN IP addresses. You can tell Apple very delibertly did not want to offer any sort of "region hopping" functionality like users require from VPNs, letting you access things like streaming content in other countries. You will always appear to be coming from your country.
3. ISP Network Information
Similar to how the TOR protocol (link) works, this will allow you to effectively hide most of what you are doing. To the ISP your traffic will simply be going to the CDN endpoint closest to you, with no DNS queries flowing to them. Those partner CDN nodes lack the complete information to connect your IP address to the request to the site. In short, it should make the information flowing across their wires much less valuable from an advertising perspective.
In terms of performance hit it should be minimal, unlike TOR. Since we are using a faster protocol with only one hop (CDN 1 -> CDN 2 -> Destination) as opposed to TOR, in my testing its pretty hard to tell the difference. While there are costs for Apple to offer the service, by limiting the traffic to just Safari, DNS and http traffic they are greatly limiting how much raw bandwidth will pass through these servers. Most traffic (like Zoom, Slack, Software Updates, etc) will all be coming from HTTPS servers.
Conclusion
Network operators, especially with large numbers of Apple devices, should take the time to read through the QUIC management document. Since the only way Apple is allowing people to "opt out" of Private Relay at a network level is by blocking DNS lookups to mask.icloud.com and mask-h2.icloud.com, many smaller shops or organizations that choose to not host their own DNS will see a large change in how traffic flows.
For those that do host their own DNS, users receive an alert that you have blocked Private Relay on the network. This is to caution you in case you think that turning it off will result in no user complaints. I won't presume to know your requirements, but nothing I've seen on the spec document for managing QUIC suggests there is anything worth blocking from a network safety perspective. If anything, it should be a maginal reduction in the amount of packets flowing across the wire.
Apple is making some deliberate choices here with Private Relay and for the most part I support them. I think it will hurt the value of some advertising and I suspect that for the months following its release the list of Apple egress nodes will confuse network operators on why they are seeing so much traffic from the same IP addresses. I am also concerned that eventually Apple will want all traffic to flow through Private Relay, adding another level of complexity for teams attempting to debug user error reports of networking problems.
From a privacy standpoint I'm still unclear on how secure this process is from Apple. Since they are controlling the encryption and key exchange, along with authenticating with the service, it seems obvious that they can work backwards and determine the IP address. I would love for them to publish more whitepapers or additional clarification on the specifics of how they handle logging around the establishment of connections.
Any additional information people have been able to find out would be much appreciated. Feel free to ping me on twitter at: @duggan_mathew.