033 - Cult of the Dead Cow

 

Imagine a world where the internet was still in its infancy, a realm waiting to be explored, manipulated, and perhaps even liberated. It was against this backdrop that a group of visionary hackers emerged, pushing the boundaries of what was possible, both technologically and ideologically.

 

We've covered a lot of hackers and the associated groups in the show. We've traveled from the emergence of networking to modern cyber warfare, but it's not very often we can follow the evolution of one group that grew up in tandem with the internet as we know it.

 

Today I want to talk a bit about secrets and the not so secrets of a group that has left an indelible mark on the digital landscape. I'm inviting you on a trip through an episode that takes us back to the genesis of cyber-activism, where code met counterculture, and innovation blended with subversion. I'm John Kordis, and for my first episode back I'm inviting you to come with me to ask the question "What the Shell is the "Cult of the Dead Cow."?

 

 

Intro

 

 

 

The Cult of the Dead Cow, often abbreviated as cDc, wasn't just a hacking group; it was a gathering of brilliant and innovative minds with a mission to challenge authority, promote free speech, and inject a healthy dose of chaos into the establishment. It's a group that's frequently associated with the genesis of the term "Hacktivism", and they've been the inspiration of many a hacker, even going so far as to be a template for some of the characters you might know or love in tv and movies.

 

In today's episode, we're going to start by traveling back to the 1980s, a time when communicating across the internet was still a challenge. Hell, access to the internet in general was difficult, so when you formed a community it meant quite a bit more. And that's something I want you to keep in mind here. I think there's this amplification of emotions, of causes, and of community that happened back then precisely because of how nontrivial it was to be present online at the time. It's not like today where I can pop online and find a subreddit, or even like it was in the 90s when you could just search for a chatroom. Finding these communities, let alone establishing them, was a challenge so when you had something like that, you held on.

 

 

Let's set the stage by talking a little bit about how a community like Cult of the Dead Cow communicated. They used what were called Bulletin Board Systems, or BBS, to establish this small community. BBS systems were the primary mode of digital communication during this age and it allowed for everything this little group of hackers could want. A typical BBS often connected through a command terminal and gave the users capabilities to do things like upload and download software, read news or zines(zeens) , and chat. So that's the framework of how the communication could operate here. What's interesting to me is recalling some of the stories I've heard about this time. You see, back then not everyone had an automatic connection. Connecting to the internet had up till around here required the use of an actual handset phone, dialing the connection number, and placing it up to the modem to open the line. It wasn't until the mid 80s that modems had developed to the point that allowed software on a computer to pick up an incoming connection, answer the phone, and then hang up the call when the connection was complete.  And, I know there are listeners of all age ranges on the show. I've talked with other industry professionals, talked to listeners who are in college, and I want to use that as a preface so I can say that I think it's necessary to bring this up. If you were confused by that line about how software would hang up the phone, I just need you to understand that for a long time phone and data ran on the same connection and it was not possible for many people to have an active internet connection and a phone call at the same time. So this development streamlined access in a way that really helped. 

 

I think I've gone off on this tangent long enough so let's get back to it. We've set the stage at 1984, that's when the Cult of the Dead Cow is believed to have been formed.  Hacker groups were forming up left and right around this time. There was, and still kind of is I guess, this kind of sci-fi romanticism about hacker underground. Depending on what you were watching or reading it offered an escape from the world proper, to a community that really just got it. Or maybe it dangled the idea of social and societal change with the application of your skill set. There seemed to be something to latch on to for everyone, whether you were a script kiddie or an elite.

 

These early days were marked quickly with notoriety for the small groups mix of technical prowess and playful antics. That initial forming of people included names that many of us know, like "Omega," "Sir Dystic," and "Oxblood", and they painted a picture of a group that was as much about pushing buttons as they were about pushing boundaries.

 

But what set the cDc apart from the myriad of hacking collectives that sprung up during this era? It was their unique blend of all of it; the tech skills…the political activism, and an unapologetic attitude really seemed to make them stand out from the crowd.. They believed in hacking for a purpose – exposing vulnerabilities, sparking public discourse, and challenging the status quo. And they did so with flair, releasing a series of hacking tools that were not just impressive in their technical execution, but also in their impact on the digital landscape. Joseph Menn, who literally wrote the book about Cult of the Dead Cow, gave a talk that touched on some of this at the 2021 Defcon Red team village and he has a point that very much liked. He described how because many kids wanted access to the internet and couldn't get it from their parent, the only way to do so was illegally. So at the get-go a lot of the hacker groups were forced to establish a moral code of where their line was because they had already crossed one to get started. This moral code and ethical boundary wasn't the same across the entire group, but it would lead a lot of the impact that it had.

 

 

Before we get into some of the impact they had, let's start with just a small intro to some of the big names that I read off earlier.

 

 

Kevin Wheeler, considered by Joseph Menn to be the groups founder. It's believed that Kevin, who grew up in Texas, started the group as he participated in the BBS boards of the time. He's often trade, share, and hoard cracked versions of software that he could take and use or distribute. As the group started to settle into something more rigid, Wheeler would end up naming it after what he believed to be quote "the unpleasant hind part of the most iconic texas industry" and thus came the name "Cult of the Dead Cow".

 

Misha Kubecka, also known as "Omega," is often considered one of the founding members of cDc. He was a key figure in the group's activities, contributing to hacking tools and writing influential texts on hacking culture.

 

Josh Buchbinder know as Sir Dystic,  was another prominent member of cDc. He was known for his technical skills and his involvement in creating one of the groups most prominent hacking tools, BackOrifice.

 

Oxblood Ruffin. He's a canadian hacker and I think that's partly why internally he was referred to as the groups "Foreign Minister".  Ruffin was one of the people here who really embraced the societal impact that hacktivism could have. Even since this point he's been active in human rights conferences as well as the typical tech conferences. Whose real name was Joel Furr, was an early member of cDc and played a crucial role in shaping the group's philosophy. He was an advocate for digital rights, privacy, and online free expression.

 

Grandmaster Ratte, who went by Swamp Ratte. His title in the cDc was "Imperial Wizard of ExXxstasy".

 

 

These are just four of the early members, we'll touch on some other members as well later on in the episode but I wanted to lay the ground work that this started as a smaller group that wasn't just about hacking for the sake of it. They had a philosophy that revolved around using technology as a tool for social change. As I mentioned earlier they were pioneers in the concept of "hacktivism," using their technical skills to challenge oppressive regimes, fight for online privacy, and highlight the importance of free expression in the digital realm. So now that we have a small glimpse as to who they are let's start to talk a little bit about what they actually did to gain the notoriety that came with it.

 

 

 

One of the first times the group really began to make ripples was in the early 90s when they decided that, much like many groups before and many after,  the church of scientology was to be their target.

 

You can actually go to the cult of the dead cow website cultdeadcow.com and find the communication that was released on June fourth 1995, and it read

 

Date: 6/4/95 7:19 PM

From: Swamp Ratte

 

FOR IMMEDIATE RELEASE

 

FOR MORE INFORMATION, CONTACT: sratte@cascade.net

 

A statement from CULT OF THE DEAD COW/cDc communications:

 

It is our belief that L. Ron "Old Mother" Hubbard is to be held

accountable

for the deaths of thousands of innocent men, women, and children in the

"Killing Fields" of Phnum Penh, as we suspect that Pol Pot was a

Scientologist!

 

We believe that El Ron Hubbard is actually none other than Heinrich Himmler

of the SS, who fled to Argentina and is now responsible for the stealing

of babies from hospitals and raising them as "super-soldiers" for the purpose

of overthrowing the U.S. Fed. Govt. in a bloody revolution.  We fear plans

for a "Fourth Reich" to be established on our home soil under the vise-like

grip of oppression known as Scientology!

 

In order to preserve our way of life and keep the torch of freedom lit

for future generations, we feel it is our duty as responsible world citizens

to declare WAR on the so-called "CHURCH" OF SCIENTOLOGY.

 

As future developments occur, we will broadcast them to the free world

as soon as possible.  Thank you for your time.

 

1. Ratte'

CULT OF THE DEAD COW/cDc communications

Fearless Leader

 

 

Ultimately not a lot is known about the campaign in terms of how serious it was and what they did. I'd imagine it was along the lines of data harvesting and service interruption but there's always the possibility it stretched even further.

 

Let's flashforward a year or so because  in 1996 a crucial piece of hacker history happened, the publishing of the essay titled "The Conscience of a Hacker" or as I think many other people know it "The Hacker Manifesto".

 

The Mentor, published this manifesto after his own arrest and it was amplified by many of the hacker zines at the time, along with the CdC. We could do a whole episode about the manifesto's impact but the down and dirty of it is that it posited that there is a point to hacking that extends beyond the selfish desires to steal, damage, exploit or harm other people, and that technology should be used to broaden our horizons. There was this idea that there was a responsibility to use hacking to try to keep the world free. In a word, hacktivism.

 

I think I'd like to continue the move up to the late 90s with what was perhaps one of their more famous accomplishments. It was the development of a tool called "Back Orifice". Back Orifice was one of the first major controversies surrounding the group.

 

The tool itself, sometimes shortened to BO was something that enabled a user to gain complete control over a computer that was running Windows. So at the time that would have been control over windows 98. Perhaps more important than that initial statement, it enabled it to happen remotely.

 

But the group didn't just drop it, they showed it at defcon that year, which would have been defcon 6. It was revealed to the public and handed out on CDs the weekend of August 1st, so they had one of the best mic drop moments in DefCon history by showing up with the easy to use tool, handing it out, and forcing Microsoft into a corner to fix their operating system.

 

See, when I was researching Sir Dystic I found his view on the tool, it was that quote

 

"BO was supposed to be a statement about the fact that people feel secure and safe, although there are wide, gaping holes in both the operating system they're using and the means of defense they're using against hostile code. I mean, that was my message…"

 

We'll regroup on B02K, the next iteration of the product in a few minutes

 

It's even been reported Sir Dystic's claim that allegedly this message was privately commended by employees of Microsoft as well.

 

The tool itself was a trojan horse kind of virus. This meant that the standard way in which it operated was that it was meant to be something disguised as a legitimate program or tool and opened by the victim.

 

9/10 times here, we're talking about phishing. Phishing, you know, the thing you have to take an annual training on every year because even now 25 years later people still don't get that you shouldn't click or open anything sent to you in an email.

 

Once it was opened a set of scripts or exploits would open the ports that allow for remote connections to the internet and allow the remote attackers who were there waiting to connect in and get even further access.

 

In their official release of the tool they would say quote “The main legitimate purposes for BO are remote tech support aid, employee monitoring and remote administering. Not that Back Orifice won't be used by overworked sysadmins, but hey, we're all adults here.

 

Back Orifice is going to be made available to anyone who takes the time to download it.”

 

And this didn't go without Microsoft trying to discredit the CdC. I want to read a bit of the interactions that occurred in the form of Microsoft's press release regarding the situation, interspersed with the CdC's rebuttal that can be found on their website.

 

I'll be flipping back and forth between what Microsoft said and what the cDc came back with. It's honestly kind of entertaining.

 

EDITING NOTE, MAYBE USE A VOICE MODULATOR FOR CDC PORTION

 

Microsoft	cDc
On July 21, a self-described hacker group known as the Cult of the Dead Cow released a tool called BackOrifice, and suggested that Windows users were at risk from unauthorized attacks.	Actually, we released it on August 3rd. Incidentally, it's been downloaded at least 35,000 times as of 11:55pm, August 7th.
Microsoft takes security seriously, and has issued this bulletin to advise customers that Windows 95 and Windows 98 users following safe computing practices are not at risk...	This is simply false. Our view is no degree of "safe computing practices" can compensate for the security bugs and lack of functionality in Windows 95 & 98.
...and Windows NT users are not threatened in any way by this tool. The Claims About BackOrifice	For the present. But remember that the tool has been around for less than a week.
According to its creators, BackOrifice is "a self-contained, self-installing utility which allows the user to control and monitor computers running the Windows operating system over a network". The authors claim that the program can be used to remotely control a Windows computer, read everything that the user types at the keyboard, capture images that are displayed on the monitor, upload and download files remotely, and redirect information to a remote internet site.	Back Orifice does not do anything that the Windows 95/98 operating system was not intended to do. It does not take advantage of any bugs in the operating system or use any undocumented or internal APIs. It uses documented calls built into Windows to do such things as: Reveal all cached passwords. This includes passwords for web sites, dialup connections, network drives and printers, and the passwords of any application that stores user passwords in the operating system. (This Windows feature was implemented apparently so the user won't be inconvenienced by having to remember his passwords every time he uses his computer.) Create shares hidden to the user and list the passwords of existing shares. Make itself mostly invisible. Back Orifice does not appear in the control-alt-delete list of running programs, and can only be killed by a low level process viewer which Windows 95 does not ship with. To their credit, Windows 98 does ship with a process viewer, but it is not installed by default.
The Truth About BackOrifice
BackOrifice does not expose or exploit any security issue with the Windows platform or the BackOffice suite of products.	Back Orifice has nothing to do, at all, with the Back Office suite. In fact, the Back Office suite only runs on NT, which isn't even supported by Back Orifice yet. Apples and Oranges.
BackOrifice does not compromise the security of a Windows network.	cDc would like to know where exactly Microsoft is getting its definition of 'compromise the security'.
Instead, it relies on the user to install it...	Back Orifice does not rely on the user for its installation. To install it, it simply needs to be run. Thanks to some actual exploits, there are several ways a program could be run on a windows computer, not only without the user's approval, but without the user's knowledge.
...and, once installed, has only the rights and privileges that the user has on the computer.	This is correct. Once installed, Back Orifice can only do what the user sitting at the computer could do, if he has programs that do everything that Back Orifice does. This includes: seeing what's on the screen seeing what's typed into the keyboard installing software uninstalling software rebooting the computer viewing stored passwords viewing and editing the system registry connecting and disconnecting the machine to other network hosts using anyone's username & password running arbitrary plugins or programs, which of course could employ any manner of exploit or attack
For a BackOrifice attack to succeed, a chain of very specific events must happen:
The user must deliberately install, or be tricked into installing the program	Not at all. Thanks to various security bugs and common system misconfigurations, there are often ways to deliver and execute arbitrary code on a Windows machine. Even lacking such an exploit, it's easy enough to provide the average Windows users a reason for downloading & installing programs from untrusted sources. It happens all the time.
The attacker must know the user's IP address.	Untrue. Back Orifice can sweep a range of IP addresses and network blocks to hunt for installations of its server software.
The attacker must be able to directly address the user's computer; e.g., there must not be a firewall between the attacker and the user.	Incorrect. The mere presence of a firewall or proxy server is not in itself a complete solution. For good, reliable protection for Windows machines on the internet, the cDc can recommend nothing better than a good, properly configured firewall. However, a firewall that permits ANY traffic is still a potential risk. Back Orifice can communicate over any available port. Therefore, if the firewall lets through any UDP packets at all, two-way communication can be established. As for file transfers originating at the remote machine, Back Orifice can use TCP to send data out through the firewall. Not to mention the hundreds of thousands of Windows 95 and 98 boxes connected to the internet via a dialed connection through their local or national isp. For mass ip vendors like those, a firewall simply isn't reasonable. Most of the internet simply wouldn't be accessible anymore.

 

And that's where I'll cut it for that portion of the content but if you look at a screenshot of the tool you can see that it would show images and local data from the impacted machine on your own display. It gave you a nice little command window that you could use to send easy commands out across the network and even let you get a glimpse into the functions you could have the machine act as which included; the built in keystroke monitor, packet sniffers, man in the middle facilitators, and more.

 

Now, this might sound bad but I'm about to make it sound worse because there's a thing we like to talk about in threat analysis called persistence. When most people not affiliated with cyber hear the word persistence I think that they think of maybe a sales person calling over and over, or maybe a lot of unwelcome advances. In this case, it means that even after a reboot, the malware would load back up and continue to operate regardless of if you removed it.

 

All in all about 15,000 copies of BO were distributed to IRC users by someone that named it nfo.zip and alleged it as being pretty useful. Unfortunately for them it was actually  the vehicle for BackOrifice. You can really see how this started to spread like wildfire, and it makes sense because of how easy it was to use and how much impact it could have for even the most early of hackers.

 

I've got a little screenshot of the tool on my website whattheshellpod.com under the episode transcript but I'll also post it to the shows Instagram for anyone to see there. You can find that at shell_pod.

 

 

 

According to Menn, when he interviewed the members there appears to have been some internal debate about the tool.

 

They knew that a lot of people would get hacked but they knew Microsoft was also feeling a bit like they could do no wrong. CdC saw a major gap in security and thought there was not going to be any security without first being compromise.

 

I wish I could say things were different now, but honestly that same kicker is still present in most companies today. If you talk to any security practitioner I think they'd agree that one of the biggest kickers to a security program always comes post breach or hack. Like it or not, but some places just will not invest until they've seen a direct impact. It's always impossible until it happens to them and here, Microsoft got an early taste first hand. Even today, Microsoft is better than they were but just last month the CEO of Tenable accused them of not giving adequate attention to security issues that were found, and again it was only when they were backed into a corner that the problem was actually addressed.

 

 

After Back Orifice was released, the group was riding a bit of a wave of public notoriety, and honestly based on what I've read, while they'd use that to further their message I genuinely don't know if they cared because that wasn't the main goal of any of their antics. I'm going to move us up to 1999 because this was when one of the defining incidents for the group would occur.

 

Let's set the stage, because at this point the internet was still pretty lawless and government entities were starting to catch up to everything you could do with this kind of access to information. One such country was China. We've talked about the Great Chinese Firewall briefly on the show in the past, but this was it's inception point. Similar to how they used their physical great wall as a shield for their people in the past, the goal the Chinese Government set out was to create a digital wall that would shield it's citizens from differing and dangerous opinions, or at least what the government deemed to be differing and dangerous.

 

The hacking community as a whole took this as an attack on freedom of information and privacy, and in particular the Cult did not approve, and it wasn't just them other groups would join in to help as well.

 

All in all, around that time there were claims of disabling a satellite, defacing Chinese government sites, and even infiltration of government computer systems. The Chinese government didn't just shrug these attacks off as children playing with computers either they ended up creating their own internet task force to monitor and try to squash attempts to spread news or activity of the hacktivists or any other kind of government opposition.

 

On December 23rd of that year, President Jiang Zemin actually threatened any of the coders and creators that were a part of this with rather large jail terms if participation continued.

 

The cult continued it's work going as far as developing an email plugin that would help normal users in China visit banned internet pages.

 

The plug in worked like a man in the middle where it would reach out and log into a proxy server, visit the web page and email back what was there to internet users in China. The thought process was that okay, you can censor what I'm seeing on the internet but if I can still receive an email and there's no introspection there, then what if I used that as my way to see the outside world. It would be like if I was in prison and the guards never checked my letters but would monitor what I put on TV in the common room. You can control what I'm doing on the TV sure, but if someone wanted to mail me pictures of the news with the current headlines, there wouldn't be anything stopping them.

 

An imperfect analogy I know, but at the time it worked out pretty well.

 

 

So let's move us up a little bit so I can talk about a couple more modern things. Because there's a lot to cover about the group and for this episode I'm just picking out what are some of my favorite highlights from the group.

 

The early 2000s marked the release of Back Orifice 2000, or BO2K depending on who you talk to. The press release read quote:

 

Back Orifice 2000 is a best-of-breed network administration tool, granting

sysadmins access to every Windows machine on their network.  Using Back

Orifice 2000, network administrators can perform typical desktop support

duties without ever leaving their desk.

 

Some notable features include:

 

    * Windows NT support (as well as Windows 95 and 98)

    * an open plugin architecture to allow for 3rd party add-ons

    * strong cryptography to ensure secure network administration

    * open source, available under the GNU Public License

 

It basically operated as an expansion and update to the original Back Orifice but with support for what was now the newest version of windows. Another blow in their battle against MS to demonstrate a lack of security.

 

Ultimately this pattern of hacktivism and tool development kept coming. There were new tools created, new efforts waged in the battle for a free and fair internet, and continuous growth for the group. And the 2010s continued to show activity increases with new members cycling in and out as you might expect. The collective even manages to continue widespread discussion on the ethics of hacking, the technical merits, and even branches off into some of the more legitimate business side of cyber security.

 

It really is interesting because you can almost see how as a collective this group has grown up with the industry. They were there when it was anarchy and the wild west and now, years later we've got people using the skills and experience in the group to try to make public and private change.

 

For example, one of the groups members Christien Rioux was able to take his skill set and help found a company called Veracode. And if that name rings a bell to you it's because it's a major player in web application security and software security. Hell, I've used the tool before and found it to be incredibly helpful when working with and educating developers. You can really see underneath the product the care and concern they have for identifying the vulnerable components of the code. But this isn't an ad for that because it's not for the regular user. I'm just trying to illustrate how the cult of the dead cow is having impact on even the mainstream.

 

The group doesn't just work in cybersecurity anymore though. As with a lot of communities, as you grow the more variance you tend to get from what kind of people join in. So you might start to see people join in the group that had aspirations beyond just being a hacker or being a presence in the cyber world.

 

That all really came to the public limelight when the American people were first introduced to Beto O'Rourke. If you look at Beto's website you'll that he self describes as a  quote

 

"fourth-generation Texan, born and raised in El Paso where he has served as a small business owner, a city council representative and a member of Congress. He founded and currently leads Powered by People, a Texas-based organization that works to expand democracy and produce Democratic victories through voter registration and direct voter engagement. Powered by People has helped register over 250,000 unregistered Texans to vote since its inception in December 2019."

 

Many of us know him from his work in the primaries and his attempt at running for president, but one thing I don't think a lot of people knew was that he was a member of "Cult of the Dead Cow". And the way this came out was as a part of Joseph Menn's book. He agreed to be one of the members that would be named as a part of this book and other corroborated his involvement with the group.

 

This is where I go back to how much variety there could be in a group as it grows because while there isn't any direct evidence suggesting Beto had any kind of involvement on the hacking side of things, he certainly has taken to heart some of the ethical messages and spirited causes that the group has endorsed.

 

Personally, I'd welcome that kind of hacker attitude into our system, but for many it never quite panned out and he'd ultimately miss his chance. I do wonder how that would have shaken out on a political stage though. Imagine if we had landed with a sitting president that had been involved in cyber action against the great firewall of China. It would have been something crazy to think about.

 

 

That year, in 2019, as I mentioned a few times already, Joseph Menn published his book aptly titled "Cult of the Dead Cow". I'm going to include a link to it in the description but it's definitely worth the read.

 

So now, let's take it to this past weekend actually. Because I think that at the time I'm recording this, Cult of the Dead Cow is getting ready to go to DefCon and have something great to share with everyone.

 

This past weekend was defcon 31. Sadly, I wasn't able to make it this year but I know there were a lot of you there so maybe you saw this.

 

But Dead Cow is showing up with a new coding framework. This is coming from a Washington Post article that was shared in the shows discord that I saw. I want to thank dawisco from our discord because they posted this and it actually prompted me to bump this episode up ahead of the original planned episode. And here's a little self promo because if you want to maybe get a mid episode credit like that, participate in our discord! The link is in the shows description and I'd love to have you.

 

So, like I was saying, the cult is planning to show up and debuted a framework for secure social networking. The post article brings up that the goal is messaging and social networking that won't keep a hold of your own personal data.

 

They're using the existence of already out there tools like Signal and Tor to try to bring the privacy back into your hands from the people that are scraping every bit of data they can on you. The key foundation is a secure end to end encryption that makes decryption and interception much harder for even nation state level attackers. Ultimately though, the tough part here as with any framework adoption is getting people on board. And I wonder if that's part of the reason we're seeing this as a Cult of the Dead Cow presentation. For lack of a better phrase they're a name brand and they bring with it credibility and fame.

 

I imagine we'll see a lot of interest coming out of this from Def Con and potentially even a lot of interference. The idea of more privacy doesn't exactly sit well with 3 letter agencies and law enforcement so it could be something that may get a bit of resistance or smearing from that side of the world.

 

I think for me the Cult of the Dead Cow was a little personal. I remember hearing kind of internet legends from that area when I was young enough to not quite understand them. I remember reading about them as I was first getting into hacking and security. And now, they're still making ripples in the space decades after they started and continuing to hold on to the very thing we started the episode discussing; a specific set of ethics and morals about it all. I'm John Kordis, and I hope this episode gave you a bit of a peak into what the shell the cult of the dead cow is, and why they're so renowned.

 

Before we go, I'm continuing one tradition I started before I took my break, and that's taking a prompt from some of the backlog that I received the first time I asked at the start of the year. This one comes from a listener named Mackenzie on Instagram and they asked me

 

"Have you ever had a personal 'uh-oh' moment where your cybersecurity skills were put to the test?"

 

So while I was scripting this out, this kind of had me wracking my head. I don't think I've had any major blunders that I can recall. I think I've landed on two stories that I can kind of tell here. One was an uh-oh that I caused in my home lab, and one was from my first job in Network Security. I think I'll start with the NetSec one. This isn't so much a blunder as a reminder that sometimes the solution in the problem is simpler than you'd think. I think I was around 23 and working in Network Security, at the time we had one kind of appliance that wasn't critical but was valuable in making sure we could manage other vendors access in and out of where they needed to be. I'm going to keep this vague because the technical details don't really matter. But I was on call, and around 1AM one Sunday morning I got a page that our appliance was offline. I signed on thinking I'd just need to make sure it failed over but the failover device was offline as well. So now, I've got no access and no way to get it because there's no response from our admin console. So, because we don't own the device I have to now page the vendor. But surprise surprise the vendor can't get access to it. I check with Network Engineering to make sure it's only our devices and after a bit of due diligence and figure out that I need drive 40 minutes to the datacenter and check on the device. Device looks alright after I plug in a KVM to check on it.  Thing is I don't have an account with any command line access. The vendor wants to troubleshoot but they're having trouble describing what they want me to do or where to start. And I kind of getting. They're another younger engineer like myself at the time and they can't see what I'm doing and we're both tired. At that point it kind of hit me, they can't see it. So long story short that's how I ended up opening my laptop, starting a webex, pointing it at a KVM and being the man in the middle for a support engineer for two hours. The problem turned out to be a bad config update on their part that accidentally switched the internal and external routing. The device couldn't route outbound and that's why we lost it. We spent way too long trying to figure out a complicated solution when ultimately it was as simple as a video chat. Looking back at it now could probably have done it better but that early morning brain fog really hurt. I think I'll tell the other one next episode so I don't drag the episode on too long, but this was one of those early instances in the field that just stuck in my head. So thanks Mackenzie for asking the question, we'll cover my home lab blunder next episode.

 

 

So that's that for this week, this was the first episode of me getting back into the swing of it. To say that I'm a little scared about returning to the form is an understatement but that's all shadowed by the excitement of it too. So if you took the time to listen, thank you. If you want to chat about this episode, please join us in the discord. I love engaging with everyone over there and the link is in the description here, on my website, and on my Instagram profile @shell_pod.

 

I'm happy to be back and I think you will be too because I've got a couple ideas for new kinds of content that will hopefully be something different than what you're used to, and help me try some fun projects. So to sign off, I'll just say thank you. Especially to those of you that are still supporting me on patreon. There were a bunch of you that I think knew I'd be coming back even before I did. So j.s , Stephen, k1lby, Frank, J, Adon, Ben Sweetnam, Ben M, John, Chris Finik, pseudo, rkaFLDvXub, last but never least, I use pot of greed to draw three additional cards from my deck….thank you. If you want to join their ranks, you can find me at patreon.com/whattheshell Let's keep this truck rolling and I'll see you all next episode.