The year was 2007, And I just started as a junior/mid/lead developer at a small web development agency called Humankind.

I've mentioned Humankind a few times, but for a deeper dive into our first product, see My First Product Launches

My first task after being hired was to fix some bugs and SaaS-ify a product we were partners on called Citizen Notification Service (CNS). It was basically a platform for towns to send out email and SMS blasts to residents about events, or emergencies. The customer was typically a town's Office of Emergency Management.

Our partner, who was a Chief of Police in a Dallas, Texas, suburb, also ran another service called CrimeWeb. CrimeWeb was a similar concept to CNS, but here the customers were police, and journalists. Basically you could sign up for CrimeWeb as a police force and disseminate crime information to journalists. The product was genius. Collect money on both ends.

After a few weeks on the job, I had fixed most of the bugs, and SaaS-ified the CNS product as best as I could with its current code-base (we would rewrite this later), and my next task was to add video upload to CrimeWeb.

Out of my depths

I'm almost never one to back down from a challenge, but CrimeWeb was written in classic ASP/VBscript, and I was an ASP.Net/C# developer, so I told the owner I wasn't completely sure if I could make this work, but he told me to try anyway, and if I couldn't get the ASP to work, maybe try to add an ASP.Net site to the classic ASP through IIS.

First step though, was to get the site working on my local environment. I grabbed the code off the server, and set it up on my local IIS. I grabbed a database backup from the server and restored it on my local SQL Server. I pointed my IIS to the code, and changed the config file, and like magic, it all worked. I was browsing around CrimeWeb.

Dressed to impress

That night after dinner, I wanted to impress my new boss, so I changed into something a little more comfortable, and continued working on the site.

I added the video upload service so that after you added a crime, the UUID was passed to the ASP.Net site, and you uploaded a video, and once it was uploaded to YouTube, the YouTube link was saved back on the crime's database record.

All that was left was to test it.

So I logged into my super administrator account, and created a new crime that was going to be limited to my local town of only 30,000 people. The title of the crime was something along the lines of "Suspected Terrorist Attack at First National Bank of Alvin. Alligators are reported fine."

The bank had Alligators... in the bank... Locally the bank was known as Alligator Bank, and the story us kids all heard is the alligators kept your money safe at night. My first bank account was there. My first 3 digits of wealth were accumulated there. And after 10 years of saving a quarter here and there, my first $100 withdraw was at their counter, in singles.

The application errored when saving the crime, and it was too late for me to investigate further.

So, I went to bed.

At around 2 in the morning, I woke up to my phone ringing. I answered, and it was my boss.

He had just spent the last hour on the phone with our partner, and he had just spent the last 2 hours answering phone calls from journalists and city officials from my town... who were apparently customers... who saw my crime posting. I felt sick. I said, there is no way for that to have happened, I edited the config, it is pointing to my local database, my browser says local host.

And then I learned about reverse proxying. It was a term I had never come across, in my many many years (2) of web development.

Apparently hidden away (on the main include file, at the very top), there was code that compared your HTTP_HOST variable to "www1" through "www10" and if it wasn't any of them, it would do a reverse proxy to a random "www1-10".

So while I had configured my site to use my local database server, and I was hosting it on my local IIS, I didn't understand the code well enough to know I wasn't on my local host.

I should have been fired

I apologized profusely to my boss, and asked him if I could at least come and get my things from the office in the morning. I didn't have much, but I had a couple of desk ornaments I had accumulated over the month and a half I had been working there.

He asked what I meant. I said that I assumed I was fired, but he said he couldn't answer that yet, as he wasn't sure what the fall out would be with our customers or our partner. I asked him if there was anything I could do. To which he replied the obvious.

I could call our partner and apologize for my colossal fuck up

I steeled myself for the worst phone call I had ever had to make... for no reason. Bob was probably the kindest and most gentle person I'd ever met. He basically chalked it up to a learning experience, told me everyone fucks up. He even gave me a great pro-tip.

When you are testing, you shouldn't report an act of terror.

Words to live by.

I wasn't fired

In fact, I stayed at Humankind for 8 very long and mostly fun years.

Through the ups and downs of the economy.

Through the ups and downs in clients.

And eventually firing all of our clients (or really not taking new ones), and working on our own products.

It was a great place to learn, and grow, and discover who I was and who I could become.