RyanCheley.com

A Giant Pain in the Ass

2026-04-06T00:00:00-07:00

Content Warning: This is a highly personal post about a cancer diagnosis.

On Feb 16, 2026 I was 'prepping' for a routine colonoscopy that was scheduled for February 17th at about 1pm. For those of you unaware what is involved in 'prepping' don't google it, but just know that your Gastroenterologist wants the 'end' of your GI tract 'clean'

This also involves a lot of not eating. So you can get hangry. Or at least most people do. I had felt something wasn't quite right for a while. Nothing I could really put my finger on, I just didn't feel right. So when it came time for the prep it turned out I wasn't really hungry anyway. I couldn't really eat the weekend before either, and had been having issues sleeping. I was stressed about what my colonoscopy would show. At least subconsciously I was worried.

On the day of my colonoscopy the staff at the office were all really nice. I even got a "First Colonoscopy" sticker!

I was wheeled into the procedure room, introduced to the doctor and told to look at the wall. The next thing I knew I was being wheeled out to the recovery room. I laid there for a few minutes and then I saw my wife Emily. I was still a bit groggy from the anesthesia but I was so happy to see her. It was the best feeling.

She came next to the bed I was laying on and the doctor came over. He let Emily know that she may want to sit down. She said she preferred to stand. The doctor then told me that during the procedure they found a tumor.

You have cancer

I let the phrase sink in ... "You have cancer" ... "I have cancer".

The doctor was not very comfortable delivering this news. You could tell this wasn't the type of thing he was used to doing. Emily even heard him saying "at the other place I don't have to tell patients this".

I think he tried his best to be positive about the diagnosis, but honestly it was a pretty shitty delivery. He kept saying things like "you're young" (at the time I was 47) ... "you're good looking" ... "you're married"

I didn't really understand why any of that mattered.

I have cancer

He then proceeded to let me know that the tumor had likely been there for years, maybe five. Had asked me if I had any symptoms, was there anything that felt off. How could I have not known something was wrong. I have cancer and it's my fault I didn't know sooner.

He also let me know that I'd need to have an ostomoy bag. For the rest of my life.

I have cancer ... that's all I could hear.

I cried. I cried in front of several people that I had never met before. I cried in front of my wife.

I have cancer. And I don't know anything about it at this point other than it seems like everyone I know with colorectal cancer died from it.

I have cancer.

I'm going to die

... those two phrases kept going through my head

I cried.

As an aside, I told a friend of mine how my cancer diagnosis was delivered to me. This friend has had many cancer battles / scares during their life. I figured when I told them my story they would say it wasn't so bad. Turns out it was. Even they were like, "Holy Shit. That's awful!"

After a few minutes, and once Emily and I were a bit more able to see the world, I was wheeled out to our car. We drove home. We didn't say much. What is there to even say? I have cancer.

We got home and although I could eat again, I wasn't hungry. I was afraid I'd never be hungry again.

That night I couldn't sleep. Or the night after. Or the night after.

The next day I had a follow up with a different GI doctor. He was basically like, "There's no new information. Why are you here?"

But we had questions. What are the next steps. Who should we contact? What do we do? The one question I didn't dare ask, "Am I going to die?"

For the next 16 days I lived in the grayest of gray areas. I could barely sleep, or eat. I lost about 8 pounds.

Emily and I spent time working to make sure that all of our affairs were in order. Are all of the banking apps installed on your phone? Do you have all of the passwords? Does the car title need to be updated? The title for the house? How do we do our finances?

We make a good team in that we each have our 'assignments'. We're pretty autonomous in those assignments. We'd talked about "cross training" on some of them, especially the financial stuff, but there just never seemed to be the time.

And now it felt like we didn't have the time but were going to have to make it. I felt like I was writing transition docs for leaving a job. But in this case I was afraid of the job I might be leaving.

To use any other word than brutal to describe these days wouldn't do justice to the way we felt. And even then it doesn't really begin to cover it.

During that time we told a few people. A very few people. Telling people made it real. Telling people was like delivering a trauma to them. Telling people led to questions. Questions we didn't have the answer for. Brutal.

I had nights where I would cry. Especially if I was alone. I have cancer, but there were things that still needed to happen. Emily had a major work event that she was responsible for. She had coworkers and friends she was able to rely on, but that didn't mean she didn't have to do things. Away from home. Away from me.

My Aunt had the same cancer diagnosis I do and she was helpful and caring and loving and kind and all of the things you need from a family member. But she didn't know the future. She didn't know if I was going to die. And so when the words, "You're going to be fine" came from her, they were nice, but hollow. I have cancer. I might die. I am scared.

On March 5th I met with a surgeon. Before meeting with the surgeon I needed to have an Abdominal CT scan done. It was completed about a week before I met with the surgeon. I had the results 2 days before meeting with the surgeon. I couldn't look at them. I didn't want to look at them.

The day of the surgery consult came. He was going to tell me the next steps. From what I heard surgery was likely to be my next step. My wife and I went to the appointment, she's been going to all of my appointments with me.

The staff were so nice and friendly and helpful. I started in one exam room and was moved to another exam room. My first thought was, "Oh no, was I in the 'you're going to be fine room' and got moved to the 'You're going to die' room? But the nurse let me know the reason for the move. A simple reason. No big deal. Except it was. It was the biggest deal. But she took the time to let me know the why of the move.

I went through the exam. Emily came back so the surgeon could talk to us.

All of the fear, and horrible anticipation. What ever he said next we were going to work through it. We were going to figure it out.

At the end of all of this, "we" might end up being just "she".

And the doctor said ...

It's actually not bad. We seem to have caught it early. We'll want to do chemo and radiation before reevaluating surgery.

I cried. This time I cried because it was the first hope I'd had in almost 3 weeks. I cried because my birthday was in 2 days and I had friends I was going to hang out with and it will be an actual happy party and not a pre wake party.

Since the surgeon I've seen a few more doctors. An oncologist and a radiation oncologist. Each appointment was mostly what one might expect. A brief conversation about potential side effects of the treatment. Which are pretty horrific if you think about them for too long. I try not to.

My treatment will be 5 days a week for 5 1/2 weeks of radiation and chemotherapy. Reevaluation of the tumor for potential surgery 6 - 12 weeks after that.

I'm sleeping better, but still not great. I eating better, but still not a ton.

And then ... for a few weeks ... nothing. Paper work is getting processed and I'm waiting for an MRI. The important part about the MRI is that it will tell me what stage and grade my tumor is. Once that's completed and the results are read then all of the doctors will have what they need to allow me to officially begin my treatment.

That being said, I have a tentative start date for my treatment. Unless my MRI shows something unexpected, I'll start my radiation and chemotherapy treatments on April 13. Officially. Fifty Five days from when I was told I had cancer to start of treatment. I'm not sure if this is a long time or not. It felt like a long time. A really long fucking time.

As part of the treatment you go in for a prep session. During this session they fit you for the device that blasts your tumor with radiation. In my case they also gave me 3 tattoos. One on either side and one right below my belly button.

I always thought my first tattoo would be of something way cooler 🤷🏻‍♂️

Before my diagnosis I had some plans for this year. I was going to go to PyCon US, PyOhio, and DjangoCon US. I even toyed with the idea of going to North Bay Python.

I won't be able to do any of these. Although my treatment will be done by late May, I'm not sure I can commit to much travel. I'm not sure how I'll be feeling.

Also, anywhere from 6 - 12 weeks after the end of my treatment I get re-evaluated for surgery. If the tumor is gone and the various docs feel like there's no risk, then surgery might not be required. If there is a risk, then surgery will be required.

The outcome of the surgery will be a colostomy bag that is either temporary (about 6 months) or permanent.

I'm less than 2 months into my cancer journey and there's still so much I don't know. Still so much that just can't be known. And honestly that's the hardest part.

My prognosis is good. My family and I are optimistic. But there's still so much we can't know. We hope that this will be a 'blip' and that by 2027 or 2028 we can go back to what ever normal is. But we just can't know.

One of the things I've really focused on over the last 2 months is trying to find the good things. I saw someone post on Mastodon (sorry, I can't find the original toot) about finding what they called glimmers. Those small things that make you happy.

I try to do that every day. A song I haven't heard in a long time. A friendly face while I'm out and about. A text from a person I haven't heard from in a while. Going for a swim. These are all things that I was taking for granted. I will likely end up taking them for granted again. But for now, I am really trying to be more appreciative of them. I'm trying to be more present.

Anyway, for those of you out there in that are 45+ and haven't gotten a colonoscopy. You should. We seem to have caught this early in the process. My prognosis is good. If someone hadn't told me I had cancer I would mostly have no idea.

Migrating to Hetzner with Coolify

2026-01-12T00:00:00-08:00

What I did

A few weeks ago, I got to watch Jeff Triplett migrate DjangoPackages from DigitalOcean to Hetzner¹ using Coolify. The magical world of Coolify made everything look just so ... easy. Jeff mentioned that one of the driving forces for the decision to go to Hetzner was the price, that is Hetzner is cheaper but with the same quality.

Aside

To give some perspective, the table below shows a comparison of what you I had and what I pay(paid) at each VPS

Server Spec	Digital Ocean Cost	Hetzner Cost	Count
Managed Database with 1GB RAM, 1vCPU	$15.15	NA	2
s-1vcpu-2gb	$12	NA	1
s-1vcpu-1gb	$6	NA	3
cpx11	NA	$4.99	2
cpx21	NA	$9.99	2
cpx31	NA	$17.99	1

With Digital Ocean I was paying about $72.50 per month for my servers. This got me 2 Managed Databases (@$15 each) and 4 Ubuntu servers (1 s-1vcpu-2gb and 3 s-1vcpu-1gb).

Based on my maths for January I should see my Hetzner bill be about $61 with the only downside being that I have to 'manage' my databases myself ... however, with Digital Ocean I always felt like I was playing with house money because I didn't have the paid for backups. Now, with Hetzner, I have backups saved to an S3 bucket (and Coolify has amazing docs for how to set this up!)

End Aside

Original State

I had 6 servers on Digital Ocean

3 production web servers
1 test web server
1 managed database production server
1 managed database test server

This cost me roughly $75 per month.

Current State

2 production web servers
1 test web server
1 production database server
1 test database server

This cost me roughly $63 per month.

Setting Up Hetzner

In order to get this all started I need to create a Hetzner account². Once I did that I created my first server, a CPX11 so that I could install Coolify.

Next, I need to clean up my DNS records. Over the years I had DNS managed at my registrar of choice (hover.com) and within Digital Ocean. Hetzner has a DNS server, so I decided to move everything there. Once all of my DNS was there, I added a record for my Coolify instance and proceeded with the initial set up.

In all I migrated 9 sites. They can be roughly broken down like this

5 Django sites
3 Datasette sites
2 Pelican site

Coolify: setting up projects, environments, resources

Coolify has several concepts that took a second to click for me

It took me some time reading through the docs but once it clicked I ended up segregating my projects in a way that made sense to me. I also ended up creating 2 environments for each project:

Production
UAT

Starting with Nixpacks

Initially I thought that Coolify would only support Docker or docker-compose files, but there is also an option for static sites, and Nixpacks. It turns out that Nixpacks were exactly what I wanted in order to get started.

NB: There is a note on the Nixpacks site that states

"⚠️ Maintenance Mode: This project is currently in maintenance mode and is not under active development. We recommend using Railpack as a replacement."

However, Railpack isn't something that Coolify offered so 🤷🏻‍♂️

I have a silly Django app called DoesTatisJrHaveAnErrorToday.com³ that seemed like the lowest risk site to start with on this experiment.

Outline of Migration steps

Allow access to database server from associated Hetzner server, i.e. Production Hetzner server needs to access Production Digital Ocean managed database server, UAT Hetzner Web server needs access to UAT Digital Ocean managed database server
Set up hetzner.domain.tld in DNS record, for example, hetzner.uat.doestatisjrhaveanerrortoday.com
Set up site in Coolify in my chosen Project, Environment, and Resource. For me this was Tatis, UAT, "Does Tatis Jr Have An Error UAT"
Configure the General tab in Coolify. For me this meant just adding an entry to 'Domains' with https://hetzner.uat.doestatisjrhaveanerrortoday.com
Configure environment variables ⁴. ⁵
Hit Deploy
Verify everything works
Update the General > Domains entry to have https://hetzner.uat.doestatisjrhaveanerrortoday.com and https://uat.doestatisjrhaveanerrortoday.com
Update DNS to have UAT point to Hetzner server
Deploy again
Wait ... for DNS propagation
Verify https://uat.doestatisjrhaveanerrortoday.com works
Remove https://hetzner.uat.doestatisjrhaveanerrortoday.com from DNS and Coolify
Deploy again
Verify https://uat.doestatisjrhaveanerrortoday.com still works
Remove the GitHub Action I had to deploy to my Digital Ocean UAT server
Repeat for Production, replacing https://hetzner.uat.doestatisjrhaveanerrortoday.com with https://hetzner.doestatisjrhaveanerrortoday.com

Repeat the process for each of my other Django sites (all 4 of them)

Switching to Dockerfile/docker-compose.yaml

This worked great, but

The highest version of Python with Nixpacks is Python 3.13
The warning message I mentioned above about Nixpacks being in "Maintenance Mode"

Also, I had a Datasette / Django app combination that I wanted to deploy, but couldn't figure out how with NIXPACKS. While the Django App isn't where I want it to be, and I'm pretty sure there's a Datasette plugin that would do most of what the Django app does, I liked the way it was set up and wanted to keep it!

Writing the `Dockerfile` and `docker-compose.yaml` files

I utilized Claude to assist with starting me off on my Dockerfile and docker-compose.yaml files. This made migrating off of the NIXPACK a bit easier than I thought it would be.

I was able to get all of my Django and Datasette apps onto a Dockerfile configuration but there was one site I have that scrape game data from TheAHL.com which has an accompanying Django app that required a docker-compose.yaml file to get set up⁶.

One gotcha I discovered was that the Coolify UI seems to indicate that you can declare your docker-compose file with any name, but my experience was that it expected the file to be called docker-compose.yaml not docker-compose.yml which did lead to a bit more time troubleshooting that I would have liked!

Upgrading all the things

OK, now with everything running from Docker I set about upgrading all of my Python versions to Python 3.14. This proved to be relatively easy for the Django apps, and a bit more complicated with the Datasette apps, but only because of a decision I had made at some point to pin to an alpha version 1.0 of Datasette. Once I discovered the underlying issue and resolved it, again, a walk in the park to upgrade.

Once I was on Python 3.14 it was another relatively straight forward task to upgrade all of my apps to Django 6.0. Honestly, Docker just feels like magic given what I was doing before and just how worried I'd get when trying to upgrade my Python versions or my Django versions.

Migrating database servers

Now I've been able to wind down all of my Web Servers, the only thing left is my managed database servers. In order to get them set up I set up pgadmin (with the help of Coolify) so that I didn't have to drop into psql in the terminal on the servers I was going to use for my database servers.

Once that was done I created backups of each database from Production and UAT on my MacBook so that I could restore them to the new Hetzner servers. To get the backup I ran this

docker run --rm postgres:17 pg_dump "postgresql://doadmin:password@host:port/database?sslmode=require" -F c --no-owner --no-privileges > database.dump

I did this for each of my 4 databases. Why did I use the docker run --rm postgres:17 pg_dump instead of just pg_dump? Because my MacBook had Postgres 16 while the server was on Postgres 17 and this was easier than upgrading my local Postgres instance.

Starting with UAT

I started with my test servers first so I could break things and have it not matter. I used my least risky site (tatis) first.

The steps I used were:

Create database on Hetzner UAT database server
Restore from UAT on Digital Ocean
Repeat for each database
Open up access to Hetzner database server for Hetzner UAT web server
Change connection string for DATABASE_URL for tatis to point to Hetzner server in my environment variables
Deploy UAT site
Verify change works
Drop database from UAT Digital Ocean database server
Verify site still runs
Repeat for each Django app on UAT
Allow access to Digital Ocean server from only my IP address
Verify everything still works
Destroy UAT managed database server
Repeat for prod

For the 8 sites / databases this took about 1 hour. Which, given how much needed to be done was a pretty quick turn around. That being said, I spent probably 2.5 hours planning it out to make sure that I had everything set up and didn't break anything, even on my test servers.

Backups

One thing about the Digital Ocean managed servers is that backups were an extra fee. I did not pay for the backups. This was a mistake ... I should have, and it always freaked me out that I didn't have them enabled. Even though these are essentially hobby projects, when you don't do the right thing you know it.

Now that I'm on non-managed servers I decided to fix that, and it turns out that Coolify has a really great tutorial on how to set up an AWS S3 bucket to have your database backups written to.

It was so easy I was able to set up the backups for each of my databases with no fuss.

Coolify dashboard thing

I mentioned above that I upgraded all of apps to Python 3.14 and Django versions to 6.0. For all of the great things about Coolify, trying to find this information out on a high level is a pain in the ass. Luckily they have a fairly robust API that allowed me to vibe code a script that would output an HTML file that showed me everything I needed to know about my Applications with respect to Python, Django, and Datasette versions. It also helped me know about my database backup setups as well⁷!

This is an example of the final state, but what I saw was some Sites on Django 4.2, others on Python 3.10 and ... yeah, it was a mess!

I might release this as a package or something at some point, but I'm not sure that anyone other than me would want to use it so, 🤷‍♂️

What this allows me to do now

One of the great features of Coolify are Preview Deployments which I've been able to implement relatively easily⁸. This allows me to be pretty confident that what I've done will work out OK. Even with a UAT server, sometimes just having that extra bit of security feels ... nice.

One thing I did (because I could, not because I needed to!) was to have a PR specific database on my UAT database server. Each database is called {project}_pr and is a full copy of my UAT database server. I have a cron job set up that restores these databases each night.

I used Claude to help generate the shell script below:

#!/bin/bash

# Usage: ./copy_db.sh source_db container_id [target_db]
# If target_db is not provided, it will be source_db_pr

source_db="$1"
container_id="$2"
target_db="$3"

# Validate required parameters
if [ -z "$source_db" ]; then
    echo "Error: Source database name required"
    echo "Usage: $0 source_db container_id [target_db]"
    exit 1
fi

if [ -z "$container_id" ]; then
    echo "Error: Container ID required"
    echo "Usage: $0 source_db container_id [target_db]"
    exit 1
fi

# Set default target_db if not provided
if [ -z "$target_db" ]; then
    target_db="${source_db}_pr"
fi

# Dynamic log filename
log_file="${source_db}_copy.log"

# Function to log with timestamp
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $*" | tee -a "$log_file"
}

log_message "Starting database copy: $source_db -> $target_db"
log_message "Container: $container_id"

# Drop existing target database
log_message "Dropping database $target_db if it exists..."
if docker exec "$container_id" psql -U postgres -c "DROP DATABASE IF EXISTS $target_db;" >> "$log_file" 2>&1; then
    log_message "✓ Successfully dropped $target_db"
else
    log_message "✗ Failed to drop $target_db"
    exit 1
fi

# Create new target database
log_message "Creating database $target_db..."
if docker exec "$container_id" psql -U postgres -c "CREATE DATABASE $target_db;" >> "$log_file" 2>&1; then
    log_message "✓ Successfully created $target_db"
else
    log_message "✗ Failed to create $target_db"
    exit 1
fi

# Copy database using pg_dump
log_message "Copying data from $source_db to $target_db..."
if docker exec "$container_id" sh -c "pg_dump -U postgres $source_db | psql -U postgres $target_db" >> "$log_file" 2>&1; then
    log_message "✓ Successfully copied database"
else
    log_message "✗ Failed to copy database"
    exit 1
fi

log_message "Database copy completed successfully"
log_message "Log file: $log_file"

Again, is this strictly necessary? Not really. Did I do it anyway just because? Yes!

Was it worth it

Hell yes!

It took time. I'd estimate about 3 hours per Django site, and 1.5 hours per non-Django site. I'm happy with my backup strategy, and preview deployments are just so cool. I did this mostly over the Christmas / New Year holiday as I fought through a cold.

Another benefit of being on Coolify is that I'm able to run

What's next?

Getting everything set up to be mostly consistent is great, but there are still some differences that exist between each Django site that don't need to when it comes to the Dockerfile that each site is using.

I also see the potential to have better alignment on the use of my third party packages. Sometimes I chose package X because that's what I knew about at the time, and then I discovered package Y but never went back and switched it out where I was using package X before.

Finally, I really want to figure out the issue of https on some of the preview deployments.

This is an affiliate link ↩︎
This is an affiliate link ↩︎
More details on why I have this site here ↩︎
Considerations for Nixpacks. The default version of Python for Nixpacks is 3.11. You can override this with an environment variable NIXPACKS_PYTHON_VERSION to allow up to Python 3.13. ↩︎
Here, you need to make sure that your ALLOWED_HOSTS is hetzner.uat.doestatisjrhaveanerrortoday.com ↩︎
Do I need this set up? Probably not. I'm pretty sure there's a Datasette plugin that does allow for edits in the SQLite database, but this was more of a Can I do this, not I need to do this kind of thing ↩︎
There are a few missing endpoints, specifically when it comes to Service database details ↩︎
one of my sites doesn't like serving up the deployment preview with SSL, but I'm working on that! ↩︎

Year in Review 2025

2025-12-31T00:00:00-08:00

I was hoping to have this written and posted last week, but for Christmas this year Santa brought me a cold which knocked me on my butt for a few days.

I had done a bit of prep, but wow, when I look back at 2025 it was a pretty big year for me personally.

Professional

I celebrated 17 years at my current employer. While this isn't a nice round number sort of anniversary, about 6 months before the actual anniversary date I was promoted to an Associated Vice President and joined the Senior Management Team. This has been a goal of mine since about 2010 and after a lot of hard work (and honestly more than a bit of good luck) I "made it".

In addition to the promotion at work, I also helped to lead a multi department team to a successful upgrade for a major application AND helped to lead a major network migration for our EHR that went really well. Two major projects accomplished in the same calendar year was a pretty good feeling.

We also do annual employee satisfaction surveys and my department had a 96% satisfaction rating. This is a really good feeling as a leader. We get shit done AND people are happy to do it!

Since 2021 my department has consistently scored above 90%. This isn't just me though! I have a great management team that helps to make this happen.

Over this same time period I've had 7 people leave the department¹. Five of them because of retirement. I really like that where I work is a place you retire at more often than not! That, along with the high satisfaction rates, suggest that my management team and I are doing something right.

Django and Python

On the Django and Python side it was also a really big year. In February I spoke at PyCascades in Portland, Oregon.

In September I spoke at DjangoCon US in Chicago, Illinois. This was my THIRD talk at DjangoCon US²

I was also active with Django Commons on the admin team, was a Djangonaut.Space Navigator in Session 5 for 2 amazing Djangonauts, and got to hang out at Jeff's Office Hours pretty consistently (though not as often as I would have liked!)

The biggest accomplishment was getting elected to the DSF Board and then being elected Treasurer of the Board. This is still all very new and I'm trying to feel my way around, but I've got some amazing support and I'm really looking forward to working with the board in 2026 and beyond.

Technology

A few weeks ago I watched Jeff Triplet migrate various infrastructure for DjangoPackages.org from Digital Ocean to Hetzner with Coolify. This got me to dive into that ... pretty deeply. I spent a lot of my December PTO³ working to migrate my servers from Digital Ocean to Hetzner managed with Coolify. I plan to write more about that later, but needless to say, as of December 29, 2025 I had successfully migrated everything off Digital Ocean to Hetzner.

Personal

Music

Watching live music is a lot of fun. My wife Emily and I really enjoy doing this. We didn't get to see as many concerts as we would have liked to, but we were still able to see a few. Kelsea Ballerini (with our daughter Abby) and Benson Boone at Crypto.com arena in Downtown Los Angeles, Sessanta at Acrisure Arena, Third Eye Blind⁴ at a local casino, a show at the Grand Ole Opry in Nashville, Tennessee as part of a conference I attended. We also saw about 5 or 6 different bands in 3 days at different venues while we were in Nashville (though sadly we didn't get to see a show on the stage at the Taco Bell Cantina which feels like a real miss!) though we did get to see The Steel Drivers at the historic Ryman Auditorium

We also saw Post Modern Jukebox at the Fox Theater in Riverside. We stayed a few days to get out of the heat of the Coachella Valley, but were shocked to learn that Riverside is only about 15-20 degrees cooler. And when it's 115-120 here it can still be above 100 there!

We still had a great time and it convinced me even more that the wild idea my friend Mario and I had to pitch Riverside as a location for DjangoCon US 2027/2028 was actually a really good idea, not just a wild idea 😀

Hockey

I went to a ton of hockey games. To start the year off I went to the Cactus Cup and saw 4 NCAA Division 1 games in 2 days. The best part was sitting behind the goal right at the glass and seeing just how fast (and LOUD) the game can be.

I was pretty exhausted (but happy) by the end of it.

I also got to see 36 Firebirds home games (regular season⁵ and post season⁶), 2 Firebirds road games (both in San Diego against the Gulls), and was able to attend the AHL All Star Competition at Acrisure Arena in February.

On the NHL front I went to a game to watch the LA Kings host the Seattle Kraken (the Firebirds big kid club) and while I was in Nashville for a conference in November I got to watch the Nashville Predators play the Calgary Flames.

My favorite hockey-related experiences this year though were the Teddy Bear Toss game (even though the Firebirds lost) and getting to greet the players in the tunnel before the game

Baseball

Sadly we only made it out to a few baseball games this year. We saw a few California Winter League games one Saturday in February, and one game out in Rancho Cucamungo to see the Dodgers Low A Affiliate the Quakes play. The Dodgers won the World Series, and that was nice, but I didn't see any of their games in person this year.

Family

This year Abby started her second year of College so Emily and I continue to be empty nesters most of the year⁷

One of my favorite highlights from the year include Abby coming home from college for the weekend for my birthday last March to surprise me. A completely unexpected visit that literally made it one of the best birthdays ever. I read something earlier this year that once your child leaves your home for college, or whatever, they'll have spent about 95% of the time they're EVER going to spend with you. This hit me pretty hard. Like wanting to sob uncontrollably hard. So for Abby to come home to spend time with me for my birthday was the best gift ever.

Conclusion

Looking back it was a pretty great year. Lots of accomplishments, lots of great memories. The year started off with lots of fear and trepidation. I still have that (in spades) but I also am starting to have a bit more hope.

I don't have any lofty goals, and I didn't do the same kind of Theme planning that I've done in the past. This year it just didn't really work for me, so I'm pausing on that exercise.

That being said, if I was going to have a theme for 2026 it would be 'The year of Intentionality'. I've spent more time this year than I would have liked doing things but not thinking about what I wanted to do. I just did them because they were easy or it's just what I always do. For the last few weeks I've been trying (with varying degrees of success) to be more intentional in my actions, and my plan is to continue that into 2026.

Here's hoping to a great 2026!

The department is 13 people ↩︎
DCUS 2023, DCUS 2024 ↩︎
Paid Time Off / Vacation ↩︎
yes, they are still a band ... no this was not my idea! ↩︎
This includes games for the 2024-25 and 2025-26 season ↩︎
just 2025, obviously ↩︎
I might just be misremembering, but when I was in college we didn't have this many breaks, and they weren't this long! ↩︎

Details on My Candidate Statement for the DSF

2025-11-04T00:00:00-08:00

The Django Software Foundation Board of Directors elections are scheduled for November 2025 and I’ve decided to throw my hat into the ring. My hope specifically, if elected, is to be selected as the Treasurer. I have 4 main objectives over my two year term.

Getting an Executive Director (ED) to help run the day-to-day operations of the DSF
Identifying small to midsized companies for sponsorships
Implementing a formal strategic planning process
Setting up a fiscal sponsorship program to allow support of initiatives like Django Commons

These are outlined in my candidate statement, but I want to provide a bit more detail why I think they’re important, and some high level details on a plan to get them to completion.

These four goals are interconnected. We need an ED to scale operations, but funding an ED requires increased revenue through corporate sponsorships. Both benefit from having a strategic plan that guides priorities. And fiscal sponsorship potentially creates a new revenue stream while strengthening the ecosystem. This isn't four separate initiatives - it's a coherent plan for sustainable growth.

Getting an Executive Director (ED) to help run the day-to-day operations of the DSF

An ED provides day-to-day operational capacity that volunteer boards simply cannot match. While board members juggle DSF work with full-time jobs, an ED could:

Call potential corporate sponsors every week, not just when someone has spare time
Coordinate directly with Django Fellows on priorities and deliverables
Support DjangoCon organizers across North & South America, Europe, Africa, and Asia with logistics and continuity
Respond to the steady stream of trademark, licensing, and community inquiries
Write grant applications to foundations that fund open source
Prepare board materials so directors can focus on governance, not research

As Jacob Kaplan-Moss says in his 2024 DjangoCon US talk

We’re already at the limit of what a volunteer board can accomplish

Right now we're missing opportunities because volunteer bandwidth is maxed out. We can't pursue major corporate sponsors that need regular touchpoints. We can't support ecosystem projects that need fiscal sponsorship. We can't scale the Fellows program even though there's clearly more work than the current Fellows can handle.

As Treasurer, hiring an ED would be my top priority. Based on comparable nonprofit ED salaries, a part-time ED (20 hours/week) would cost approximately $60,000-$75,000 annually including benefits and overhead. A full-time ED would be $120,000-$150,000.

The DSF's current annual budget is roughly $300,000. Adding even a part-time ED would require increasing revenue by 25-30%. This is exactly why my second priority focuses on corporate sponsorships - we need sustainable revenue growth to support professional operations.

The path forward is phased: board members initiate corporate outreach to fund a part-time ED, who then scales up fundraising efforts to eventually become full-time and bring us toward that $1M budget Jacob outlined. We bootstrap professional operations through volunteer effort, then let the professional accelerate what volunteers started.

Identifying small to midsized companies for sponsorships

In Jacob Kaplan-Moss' 2024 DjangoCon US Talk, he outlines what the DSF could do with a $1M budget. I believe this is achievable, but it requires a systematic approach to corporate sponsorships.

Currently, the DSF focuses primarily on major sponsors. This makes sense - volunteer boards have limited bandwidth, so targeting "whales" is efficient. But we're leaving significant revenue on the table.

Consider the numbers: US Census data shows roughly 80,000-400,000 small to mid-sized tech companies (depending on definitions). Stack Overflow's 2024 survey indicates 46.9% of professional developers use Python, and 38% of Python web developers use Django. Even capturing a small fraction of companies using Django in production at a modest sponsorship tier ($500-$2,500/year) could significantly increase DSF revenue.

The challenge isn't identifying companies - it's having capacity to reach them. This is where an Executive Director becomes critical.

What an Executive Director Would Enable

A part-time Executive Director (ED) could:

Dedicate 10+ hours weekly to corporate outreach instead of the 1-2 hours volunteer board members can spare
Maintain a CRM system tracking sponsor relationships, touchpoints, and renewal cycles
Create targeted outreach campaigns to Django-using companies in specific sectors (healthcare tech, fintech, e-commerce, etc.)
Develop case studies showing Django's business value to help companies justify sponsorship
Provide consistent follow-up and relationship management that volunteer boards cannot maintain

My First 90 Days as Treasurer

If elected, here's my concrete plan:

Month 1:

Audit current sponsors and revenue sources
Identify 20 target companies (mix of sizes) currently using Django
Work with current board to draft outreach templates and sponsorship value propositions

Month 2:

Begin systematic outreach to target companies
Track response rates and refine approach
Engage with Django community leaders to identify additional prospects

Month 3:

Report results to board
If we've secured commitments for additional $30K-$50K in annual recurring revenue, propose budget to hire part-time ED
Continue to push forward the ED recruitment process

This is realistic volunteer-level effort (5-8 hours/week) that proves the concept before committing to an ED hire. Once we have an ED, they can scale this 5-10x.

Implementing a formal strategic planning process

The DSF needs a strategic plan - not as a bureaucratic exercise, but as a practical tool for making decisions and measuring progress.

Right now, we operate somewhat reactively. The Fellows program exists because it was created years ago. DjangoCons happen because organizers step up. Corporate sponsorships come in when companies reach out to us. This isn't necessarily bad, but it means we're not proactively asking: What should Django's ecosystem look like in 5 years? How do we get there?

A strategic plan would give us:

Clear priorities: When opportunities arise (a major donor, a new initiative, a partnership proposal), we can evaluate them against stated goals rather than deciding ad-hoc.

Accountability: We can measure whether we're making progress on what we said mattered. Did we grow the Fellows program like we planned? Did sponsorship revenue increase as projected?

Communication: Community members and potential sponsors can understand where the DSF is headed and how they can contribute.

As someone who's been in healthcare management since 2012, I've seen how strategic planning drives organizational effectiveness. The best plans aren't 50-page documents that sit on a shelf - they're living documents that inform quarterly board discussions and annual budget decisions.

For the DSF, I envision a strategic planning process that:

Year 1:

Conduct stakeholder interviews with Fellows, corporate sponsors, community leaders, and DjangoCon organizers
Identify 3-5 strategic priorities for the next 3 years (e.g., "double sponsorship revenue," "launch fiscal sponsorship program," "expand geographic diversity of Django community")
Develop measurable outcomes for each priority
Share draft plan with community for feedback

Ongoing:

Review progress quarterly at board meetings
Publish annual progress reports
Revise plan every 3 years based on outcomes and changing needs

This connects directly to my other goals: we need a strategic plan to guide ED hiring, fundraising priorities, and fiscal sponsorship criteria. Without it, we're making isolated decisions rather than building toward a coherent vision.

Setting up a fiscal sponsorship program to allow support of initiatives like Django Commons

Django's success isn't just about the framework itself. It's about the ecosystem of packages, tools, and community organizations that have grown around it. Projects like Django Commons, Django Packages, regional Django user groups, and specialized packages serve thousands of developers daily. Yet these projects face a common challenge: they lack the legal and financial infrastructure to accept donations, pay for infrastructure, or compensate maintainers.

A fiscal sponsorship program would allow the DSF to serve as the legal and financial home for vetted Django ecosystem projects. Think of it as the DSF saying: "We'll handle the paperwork, taxes, and compliance; you focus on serving the community."

Who This Helps

Community maintainers who need to accept donations but shouldn't have to become nonprofit experts
Django Commons and similar initiatives that need to pay for infrastructure, security audits, or maintainer stipends
Regional Django organizations that want to organize events or workshops but lack financial infrastructure
Critical packages in the Django ecosystem that need sustainable funding models
Corporate sponsors who want to support the broader ecosystem but need a tax-deductible vehicle

Why This Matters

Right now, valuable Django ecosystem projects are essentially flying without a net. If Django Commons needs to accept a $10,000 corporate donation to fund security audits, there's no clear path to doing so. If a critical package needs to pay for CI/CD infrastructure or compensate a maintainer for urgent security fixes, they're stuck. Some projects, such as Djangonaut Space, have tried to solve this individually by creating their own 501(c)(3)s or using platforms like Open Collective, but this fragments the community and creates overhead.

The Python Software Foundation already does this successfully for PyPI, PyLadies, and regional Python conferences. NumFOCUS sponsors dozens of scientific Python projects. There's no reason Django's ecosystem shouldn't have similar support.

For the DSF, this is also about long-term sustainability. A healthy Django depends on a healthy ecosystem. When popular packages go unmaintained or community initiatives shut down due to funding constraints, Django suffers. By providing fiscal sponsorship, we strengthen the entire Django community while also creating a new (modest) revenue stream through administrative fees that can fund DSF operations.

Moving Forward Together

These four initiatives - (1) hiring an Executive Director, (2) growing corporate sponsorships, (3) implementing strategic planning, and (4) launching fiscal sponsorship - represent an ambitious but achievable vision for the DSF's next two years. They're not just ideas; they're a roadmap for taking Django from a volunteer-run project to a professionally-supported ecosystem that can serve millions of developers for decades to come.

If you believe in this vision and think I can help make it happen, I'd be honored to have your vote. You can find more about my background and community involvement in my candidate statement.

Thank you for taking the time to read this, and regardless of the election outcome, I'm committed to supporting Django's continued success.

Deploying n8n on Digital Ocean

2025-10-11T00:00:00-07:00

This guide shows you how to deploy n8n, a workflow automation tool, on your own VPS. Self-hosting gives you full control over your data, avoids monthly subscription costs, and lets you run unlimited workflows without usage limits.

I'm using Digital Ocean¹ for this guide, but these steps work on any VPS provider. You'll need:

A VPS with Ubuntu 24.04 (minimum 1GB RAM)
A domain name with DNS access
Basic familiarity with SSH and command line tools

Create and configure the VPS

Create a droplet with Ubuntu 24.04. Select a plan with at least:

1GB RAM
25GB Disk
1 vCPU

Note the IP address - you'll need it for DNS configuration.

SSH into the server:

ssh root@ipaddress

Update the system:

apt update
apt upgrade -y

Install Docker

Install Docker using the official repository:

# Add Docker's official GPG key
sudo apt-get update
sudo apt-get install ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc

# Add the repository to Apt sources
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
  $(. /etc/os-release && echo "${UBUNTU_CODENAME:-$VERSION_CODENAME}") stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update

# Install Docker and its components
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

Configure DNS

Create an A record at your domain registrar pointing your subdomain (e.g., n8n.yourdomain.com) to your droplet's IP address. If you're using Hover, follow their DNS management guide.

Create Docker Compose configuration

Create a docker-compose.yml file on your server. Start with the Caddy service for handling SSL and reverse proxy:

services:
  caddy:
    image: caddy:latest
    ports:
      - "80:80"
      - "443:443"
    restart: always
    volumes:
      - ./Caddyfile:/etc/caddy/Caddyfile:ro
      - caddy_data:/data
      - caddy_config:/config
      - ./logs:/var/log/caddy
    deploy:
      resources:
        limits:
          cpus: '0.5'
          memory: 500M
    healthcheck:
      test: ["CMD", "caddy", "version"]
      interval: 30s
      timeout: 10s
      retries: 3
    logging:
      driver: "json-file"
      options:
        max-size: "10m"
        max-file: "3"
volumes:
  caddy_data:
  caddy_config:

Create a Caddyfile in the same directory, replacing n8n.mydomain.com with your actual domain:

n8n.mydomain.com {
    # Enable compression
    encode gzip zstd

    # Reverse proxy to n8n
    reverse_proxy n8n:5678 {
        header_up Host {host}
        header_up X-Real-IP {remote}
        header_up X-Forwarded-For {remote}
        header_up X-Forwarded-Proto {scheme}
        header_up X-Forwarded-Host {host}

        transport http {
            keepalive 30s
            keepalive_idle_conns 10
        }
        flush_interval -1
    }

    # Security headers (relaxed CSP for n8n's dynamic interface)
    header {
        Strict-Transport-Security "max-age=31536000; includeSubDomains; preload"
        X-Content-Type-Options "nosniff"
        X-Frame-Options "SAMEORIGIN"
        Referrer-Policy "strict-origin-when-cross-origin"
        Content-Security-Policy "default-src 'self'; script-src 'self' 'unsafe-inline' 'unsafe-eval'; style-src 'self' 'unsafe-inline'; img-src 'self' data: blob:; connect-src 'self' wss: ws:; frame-src 'self'; worker-src 'self' blob:;"
        -Server
    }

    # Enable logging
    log {
        output file /var/log/caddy/n8n-access.log {
            roll_size 10MB
            roll_keep 5
        }
        format json
    }

    # Enable TLS with reasonable settings
    tls {
        protocols tls1.2 tls1.3
    }
}

Add n8n to Docker Compose

Add the n8n service under services: in your docker-compose.yml file. Replace n8n.mydomain.com with your domain in the environment variables:

  n8n:
    image: n8nio/n8n:latest
    container_name: n8n
    restart: always
    environment:
      - N8N_HOST=n8n.mydomain.com
      - N8N_PORT=5678
      - WEBHOOK_URL=https://n8n.mydomain.com/
      - GENERIC_TIMEZONE=UTC
    ports:
      - "5678:5678"
    volumes:
      - n8n_data:/home/node/.n8n
      - /etc/timezone:/etc/timezone:ro
      - /etc/localtime:/etc/localtime:ro
    deploy:
      resources:
        limits:
          cpus: '1.0'
          memory: 1G
    healthcheck:
      test: ["CMD", "wget", "--no-verbose", "--tries=1", "--spider", "http://localhost:5678/healthz"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 60s
    logging:
      driver: "json-file"
      options:
        max-size: "10m"
        max-file: "3"

Add n8n_data: to the volumes: section in your docker-compose.yml file:

volumes:
  caddy_data:
  caddy_config:
  n8n_data: # new line

Your final docker-compose.yml file will look like this:

services:
  caddy:
    image: caddy:latest
    ports:
      - "80:80"
      - "443:443"
    restart: always
    volumes:
      - ./Caddyfile:/etc/caddy/Caddyfile:ro
      - caddy_data:/data
      - caddy_config:/config
      - ./logs:/var/log/caddy
    deploy:
      resources:
        limits:
          cpus: '0.5'
          memory: 500M
    healthcheck:
      test: ["CMD", "caddy", "version"]
      interval: 30s
      timeout: 10s
      retries: 3
    logging:
      driver: "json-file"
      options:
        max-size: "10m"
        max-file: "3"
  n8n:
    image: n8nio/n8n:latest
    container_name: n8n
    restart: always
    environment:
      - N8N_HOST=n8n.mydomain.com
      - N8N_PORT=5678
      - WEBHOOK_URL=https://n8n.mydomain.com/
      - GENERIC_TIMEZONE=UTC
    ports:
      - "5678:5678"
    volumes:
      - n8n_data:/home/node/.n8n
      - /etc/timezone:/etc/timezone:ro
      - /etc/localtime:/etc/localtime:ro
    deploy:
      resources:
        limits:
          cpus: '1.0'
          memory: 1G
    healthcheck:
      test: ["CMD", "wget", "--no-verbose", "--tries=1", "--spider", "http://localhost:5678/healthz"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 60s
    logging:
      driver: "json-file"
      options:
        max-size: "10m"
        max-file: "3"
volumes:
  caddy_data:
  caddy_config:
  n8n_data:

Start the containers

Run the containers in detached mode:

docker compose up -d

Complete the setup

Navigate to https://n8n.yourdomain.com in your browser. Follow the setup wizard to create your admin account. Once complete, you can start building workflows.

Referral Link ↩︎

Why We Need to Stop Fighting About AI Tools and Start Teaching Them

2025-07-25T00:00:00-07:00

In mid-June, Hynek tooted on Mastodon the following toot:

Watching the frustratingly fruitless fights over the USEFULNESS of LLM-based coding helpers, I've come down to 3 points that explain why ppl seem to live in different realities:

Most programmers:

1) Write inconsequential remixes of trivial code that has been written many times before.

2) Lack the taste for good design & suck at code review in general (yours truly included).

3) Lack the judgement to differentiate between 1) & FOSS repos of nontrivial code, leading to PR slop avalanche.

1/3

So, if you're writing novel code & not another CRUD app or API wrapper, all you can see is LLMs fall on their faces.

Same goes for bigger applications if you care about design. Deceivingly, if you lack 2), you won't notice that an architecture is crap b/c it doesn't look worse than your usual stuff.

That means that the era of six figures for CRUD apps is coming to an end, but it also means that Claude Code et al can be very useful for certain tasks. Not every task involves splitting atoms. 2/3

2/3

There's also a bit of a corollary here. Given that LLMs are stochastic parrots, the inputs determine the outputs.

And, without naming names, certain communities are more… rigorous… at software design than others.

It follows that the quality of LLM-generated code will inevitably become a decision factor for choosing frameworks and languages and I'm not sure if I'm ready for that.

3/3

I've been having a lot of success with using Claude Code recently so I've been thinking about this toot a lot lately. Simon Willison talks a lot about the things that he's been able to do because he just asks OpenAI's ChatGPT while walking his dog. He's asking a coding agent to help him with ideas he has in languages with which he may not be familiar. However, he's a good enough programmer that he can spot anti-patterns that are being written by the agent.

For me, it comes down to the helpfulness of these agentic coding tools; they can help me write boiler plate code more quickly. What it's really coming down to, for me, is that when something is trivially easy to implement, like another CRUD app or an API wrapper, those problems are solved. We don't need to keep solving them in ways that don't really help. What we need to do in order to be better programmers is figure out how to solve problems most effectively. And if that's creating a CRUD app or an API wrapper or whatever, then yeah, you're not solving any huge problem there. But if you're looking to solve something in a very unique or novel way, agentic coding tools aren't going to help you as much.

I don't need to know how the internal combustion engine of my car works. I do need to know that when the check engine light comes on, I need to take it to a mechanic. And then that mechanic is going to use some device that lets them know what is wrong with the car and what needs to be done to fix it. This seems very analogous to the coding agents that we're seeing now. We don't have to keep trying to solve those problems with well-known solutions. We can and we should rely on the knowledge that is available to us and use that knowledge to solve these problems quickly. This allows us to focus on trying to solve new problems that no one has ever seen.

This doesn't mean we can skip learning the fundamentals. Like blocking and tackling in football, if you can't handle the basic building blocks of programming, you're not going to succeed with complex projects. That foundational understanding remains essential.

The real value of large language models and coding agents lies in how they can accelerate that learning process. Being able to ask an LLM about how a specific GitHub action works, or why you'd want to use a particular pattern, creates opportunities to understand concepts more quickly. These tools won't solve novel problems for you—that's still the core work of being a software developer. But they can eliminate the repetitive research and boilerplate implementation that used to consume so much of our time, freeing us to focus on the problems that actually require human creativity and problem-solving skills.

How many software developers write in assembly anymore? Some of us maybe, but really what it comes down to is that we don't have to. We've abstracted away a lot of that particular knowledge set to a point where we don't need it anymore. We can write code in higher-level languages to help us get to solutions more quickly. If that's the case, why shouldn't we use LLMs to help us get to solutions even more quickly?

I've noticed a tendency to view LLM-assisted coding as somehow less legitimate, but this misses the opportunity to help developers integrate these tools thoughtfully into their workflow. Instead of questioning the validity of using these tools, we should be focusing on how we can help people learn to use them effectively.

In the same way that we helped people to learn how to use Google, we should help them to use large language models. Back in the early 2000s when Google was just starting to become a thing, knowing how to effectively use it to exclude specific terms, search for exact phrases using quotation marks, that wasn't always known by everybody. But the people who knew how to do that were able to find things more effectively.

I see a parallel here. Instead of dismissing people who use these tools, we should be asking more constructive questions: How do we help them become more effective with LLMs? How do we help them use these tools to actually learn and grow as developers?

Understanding the limitations of large language models is crucial to using them well, but right now we're missing that opportunity by focusing on whether people should use them at all rather than how they can use them better.

We need to take a step back and re-evaluate how we use LLMs and how we encourage others to use them. The goal is getting to a point where we understand that LLMs are one more tool in our developer toolkit, regardless of whether we're working on open-source projects or commercial software. We don't need to avoid these tools. We just need to learn how to use them more effectively, and we need to do this quickly.

Migrating to Raindrop.io

2025-06-19T00:00:00-07:00

With the announced demise of Pocket by Mozilla I needed to migrate all of my saved articles to 'something else' by the end of the month. I've actually tried to migrate from Pocket a few times over the years. I landed on Instapaper for a while, but it never really clicked for me. I tried a service called Devmarks that Adam G Hill runs, and I really liked it, but for whatever reason I stopped using it. I had also previously tried Raindrop.io ... and I'm not really sure what drove me away from it, but it didn't stick for me at the time.

Since I didn't have a choice about Pocket I did a bit of purusing my options, and finally landed on Raindrop.io again. The process of migration is pretty painless. I just export out the links from Pocket and then import them into Raindrop. No fuss ... no muss. Raindrop even checks for duplicates and allows you to not import them!

So, I imported everything (all 11,500+ articles!) and started to incorporate Raindrop into my workflow. This basically just means saving things to Raindrop instead of pocket, and then checking Raindrop instead of Pocket every week to make sure I'm all caught up on my articles to read.

Over the last weekend I was looking at how all of the imported items in Raindrop were put into the 'archive' collection and decided that I could probably do something about putting them into proper collections.

With the help of Claude Code, I was able to put them into better collections. There were some stragglers and I decided that I could categorize them on my own (there were less than 100).

I started going through these last ones I kept coming across articles for iOS7, or an app that I think I liked in 2015 but isn't on the App store anymore. I came across this article (which I also tooted about on Mastadon) from September 4, 2014 with the title What the Internet of Things Will Look Like in 2025 (Infographic). It's wildly naive, but a fun read nonetheless.

Needless to say it was the only gem in the 100 articles that I went through. I had so many saved articles that aren't 'Evergreen'. I then started looking at some of the articles that had been categorized and came across stuff for Django 1.11, Python 3.8, and other older stuff.

These were great articles when I read them, but I don't know that I need them now. In fact, when I looked at my general workflow for using any read-it-later service, I essentially save it to read later. If it's sitting in my read-it-later service for more than 4 weeks I'll either delete or just archive it.

So really, unless I'm planning on doing something with these articles, I'm not sure that I need to keep them. And that's when it hit me ... I can just delete them. All of them. I don't need to keep them. If they are truly impactful, I can write up something about them in Obsidian. If I really think someone else will get something out of my reaction I can write it up and post it. But, if I'm being honest with myself, this is just digital clutter that isn't "sparking" any joy for me.

So, just like that, I went from having 11,000+ links to having 0. And I have no ragrets.

I'm sure there's some deeper story here about physical things and just letting them go as well, and maybe I'll be able to apply that to my non-digital life, but for now, I'm just going to revel in the fact that I was able to offload this thing and just not ... care? Be sad? I'm not sure what the correct term would be here.

Regardless, it was a good exercise to have gone through, and I'm glad I did.

A New Project at Work

2025-06-15T00:00:00-07:00

I was added to a work email that was requesting a not-so-small new project that was going to need to be completed. The problem that needed to be solved was a bit squishy, but it had been well thought out, and it had an importance to it that was easy to see.

There was still some workflows and data that needed to be reviewed, but overall it was on a good path to having a real project feel to it.

One question still outstanding is, what platform will this project be implemented on? In our EHR, or on a separate web app?

During my weekly project review meeting with the Web Development team I let them know about the potential for this new project and that it would likely need to take priority over one of our current projects. The start is still a couple of weeks away so we have time to plan for it (as much as we can anyway). We looked at the project board and determined a ranking of the current projects. We decided on the project that would likely get bumped if this new one ends up with the web developers. And just like that we had a contingency plan for how to plan for this project given our current constraints.

Now, this project may never make its way to the web development team, but having that conversation with the manager, and then during our standup today, to let the team know that this might be something that will need to be worked on by them felt right. No surprises in a few weeks. No randomness about what projects we'll be working on ... just a bit of planning to prepare for something that might never come.

Eisenhower said, "Plans are nothing, planning is everything."

The team appreciated being in the loop about a potential project and being able to align expectations moving forward. I felt grateful that this was brought to my attention well before it was submitted as a request. The requester now has a bit more information on who to speak with internally, and it really felt like we were working together to solve a problem in a very professional way.

I wish all projects started like this. It would make life way easier and not so much like this

Updated TIL

2025-06-05T00:00:00-07:00

While browsing Mastodon the other day I came across a toot that linked to this Cursed Knowledge page. I thought it was a great page, but it occurred to me that it could be helpful to apply the same sort of styling to my TIL Repo.

My TIL (Today I Learned) repository was inspired by Simon Wilison's and is ~~basically a rip off~~ completely inspired by him.

Now, I'm not a front end guy, and while I can do some pretty basic stuff with GitHub Actions, I knew that what I wanted to do far outstripped my current capabilities. Enter Claude!

I started with this prompt

You are an expert in GitHub Actions, and GitHub. Additionally, you are an expert in HTML, CSS, and JavaScript. The solutions provided can not leverage JavaScript frameworks.

I then made this request

I currently have a Today I learned (TIL) repository on GitHub located here

https://github.com/ryancheley/til

I came across the idea of documenting cursed information here

https://immich.app/cursed-knowledge/

I would like to implement my TIL in a timeline the same way the cursed knowledge page was done.

How can I do that?

It output this

name: Build TIL Timeline

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]
  schedule:
    # Rebuild daily at 6 AM UTC
    - cron: '0 6 * * *'

permissions:
  contents: read
  pages: write
  id-token: write

jobs:
  build-timeline:
    runs-on: ubuntu-latest

    steps:
    - name: Checkout repository
      uses: actions/checkout@v4

    - name: Setup Node.js
      uses: actions/setup-node@v4
      with:
        node-version: '18'

    - name: Generate timeline data
      run: |
        node << 'EOF'
        const fs = require('fs');
        const path = require('path');

        // Function to extract metadata from markdown files
        function extractTilData(filePath, content) {
          const relativePath = path.relative('.', filePath);
          const fileName = path.basename(filePath, '.md');

          // Extract title from first heading or use filename
          const titleMatch = content.match(/^#\s+(.+)$/m);
          const title = titleMatch ? titleMatch[1] : fileName.replace(/[-_]/g, ' ');

          // Extract first paragraph as description
          const lines = content.split('\n').filter(line => line.trim());
          let description = '';
          for (const line of lines) {
            if (!line.startsWith('#') && line.trim() && !line.startsWith('```')) {
              description = line.trim();
              break;
            }
          }

          // Get file stats for date
          const stats = fs.statSync(filePath);
          const date = stats.mtime; // Use modification time

          // Extract category from directory structure
          const pathParts = relativePath.split('/');
          const category = pathParts.length > 1 ? pathParts[0] : 'General';

          return {
            title,
            description: description || 'No description available',
            date: date.toISOString().split('T')[0], // YYYY-MM-DD format
            url: `https://github.com/ryancheley/til/blob/main/${relativePath}`,
            category,
            fileName: relativePath
          };
        }

        // Recursively find all markdown files
        function findMarkdownFiles(dir) {
          const files = [];
          const entries = fs.readdirSync(dir, { withFileTypes: true });

          for (const entry of entries) {
            const fullPath = path.join(dir, entry.name);
            if (entry.isDirectory() && !entry.name.startsWith('.')) {
              files.push(...findMarkdownFiles(fullPath));
            } else if (entry.isFile() && entry.name.endsWith('.md') && entry.name !== 'README.md') {
              files.push(fullPath);
            }
          }

          return files;
        }

        // Generate timeline data
        const markdownFiles = findMarkdownFiles('.');
        const tilEntries = [];

        for (const filePath of markdownFiles) {
          try {
            const content = fs.readFileSync(filePath, 'utf8');
            const tilData = extractTilData(filePath, content);
            tilEntries.push(tilData);
          } catch (error) {
            console.error(`Error processing ${filePath}:`, error.message);
          }
        }

        // Sort by date (newest first)
        tilEntries.sort((a, b) => new Date(b.date) - new Date(a.date));

        // Create output directory
        if (!fs.existsSync('docs')) {
          fs.mkdirSync('docs');
        }

        // Write timeline data
        fs.writeFileSync('docs/timeline-data.json', JSON.stringify(tilEntries, null, 2));

        console.log(`Generated timeline with ${tilEntries.length} entries`);
        EOF

    - name: Generate timeline HTML
      run: |
        mkdir -p docs
        cat > docs/index.html << 'EOF'
        <!DOCTYPE html>
        <html lang="en">
        <head>
            <meta charset="UTF-8">
            <meta name="viewport" content="width=device-width, initial-scale=1.0">
            <title>Ryan's Today I Learned Timeline</title>
            <style>
                * {
                    margin: 0;
                    padding: 0;
                    box-sizing: border-box;
                }

                body {
                    font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', system-ui, sans-serif;
                    background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
                    min-height: 100vh;
                    color: #333;
                }

                .container {
                    max-width: 1200px;
                    margin: 0 auto;
                    padding: 2rem;
                }

                .header {
                    text-align: center;
                    margin-bottom: 3rem;
                    color: white;
                }

                .header h1 {
                    font-size: 3rem;
                    margin-bottom: 1rem;
                    text-shadow: 2px 2px 4px rgba(0,0,0,0.3);
                }

                .header p {
                    font-size: 1.2rem;
                    opacity: 0.9;
                    max-width: 600px;
                    margin: 0 auto;
                }

                .timeline {
                    position: relative;
                    margin-top: 2rem;
                }

                .timeline::before {
                    content: '';
                    position: absolute;
                    left: 2rem;
                    top: 0;
                    bottom: 0;
                    width: 2px;
                    background: linear-gradient(to bottom, #4CAF50, #2196F3, #FF9800, #E91E63);
                }

                .timeline-item {
                    position: relative;
                    margin-bottom: 2rem;
                    margin-left: 4rem;
                    background: white;
                    border-radius: 12px;
                    padding: 1.5rem;
                    box-shadow: 0 8px 25px rgba(0,0,0,0.1);
                    transition: transform 0.3s ease, box-shadow 0.3s ease;
                }

                .timeline-item:hover {
                    transform: translateY(-5px);
                    box-shadow: 0 15px 35px rgba(0,0,0,0.15);
                }

                .timeline-item::before {
                    content: '';
                    position: absolute;
                    left: -3rem;
                    top: 2rem;
                    width: 16px;
                    height: 16px;
                    background: #4CAF50;
                    border: 3px solid white;
                    border-radius: 50%;
                    box-shadow: 0 0 0 3px rgba(76, 175, 80, 0.3);
                }

                .timeline-item:nth-child(4n+2)::before { background: #2196F3; box-shadow: 0 0 0 3px rgba(33, 150, 243, 0.3); }
                .timeline-item:nth-child(4n+3)::before { background: #FF9800; box-shadow: 0 0 0 3px rgba(255, 152, 0, 0.3); }
                .timeline-item:nth-child(4n+4)::before { background: #E91E63; box-shadow: 0 0 0 3px rgba(233, 30, 99, 0.3); }

                .timeline-header {
                    display: flex;
                    justify-content: space-between;
                    align-items: flex-start;
                    margin-bottom: 1rem;
                    flex-wrap: wrap;
                    gap: 1rem;
                }

                .timeline-title {
                    font-size: 1.4rem;
                    font-weight: 600;
                    color: #2c3e50;
                    text-decoration: none;
                    flex-grow: 1;
                    transition: color 0.3s ease;
                }

                .timeline-title:hover {
                    color: #3498db;
                }

                .timeline-meta {
                    display: flex;
                    gap: 1rem;
                    align-items: center;
                    flex-shrink: 0;
                }

                .timeline-date {
                    background: linear-gradient(135deg, #667eea, #764ba2);
                    color: white;
                    padding: 0.5rem 1rem;
                    border-radius: 20px;
                    font-size: 0.9rem;
                    font-weight: 500;
                }

                .timeline-category {
                    background: #f8f9fa;
                    color: #6c757d;
                    padding: 0.4rem 0.8rem;
                    border-radius: 15px;
                    font-size: 0.8rem;
                    font-weight: 500;
                    border: 1px solid #e9ecef;
                }

                .timeline-description {
                    color: #5a6c7d;
                    line-height: 1.6;
                    font-size: 1rem;
                }

                .loading {
                    text-align: center;
                    padding: 3rem;
                    color: white;
                    font-size: 1.2rem;
                }

                .error {
                    background: #f8d7da;
                    color: #721c24;
                    padding: 1rem;
                    border-radius: 8px;
                    border: 1px solid #f5c6cb;
                    margin-bottom: 2rem;
                }

                .stats {
                    background: rgba(255,255,255,0.95);
                    border-radius: 12px;
                    padding: 1.5rem;
                    margin-bottom: 2rem;
                    backdrop-filter: blur(10px);
                    border: 1px solid rgba(255,255,255,0.2);
                }

                .stats-grid {
                    display: grid;
                    grid-template-columns: repeat(auto-fit, minmax(200px, 1fr));
                    gap: 1rem;
                    text-align: center;
                }

                .stat-item {
                    padding: 1rem;
                }

                .stat-number {
                    font-size: 2rem;
                    font-weight: bold;
                    color: #667eea;
                    display: block;
                }

                .stat-label {
                    color: #666;
                    font-size: 0.9rem;
                    margin-top: 0.5rem;
                }

                @media (max-width: 768px) {
                    .container {
                        padding: 1rem;
                    }

                    .header h1 {
                        font-size: 2rem;
                    }

                    .timeline::before {
                        left: 1rem;
                    }

                    .timeline-item {
                        margin-left: 2rem;
                        padding: 1rem;
                    }

                    .timeline-item::before {
                        left: -2rem;
                    }

                    .timeline-header {
                        flex-direction: column;
                        align-items: stretch;
                    }

                    .timeline-meta {
                        justify-content: space-between;
                    }
                }
            </style>
        </head>
        <body>
            <div class="container">
                <div class="header">
                    <h1>Today I Learned</h1>
                    <p>A timeline of discoveries, learnings, and insights from my development journey</p>
                </div>

                <div class="stats" id="stats" style="display: none;">
                    <div class="stats-grid">
                        <div class="stat-item">
                            <span class="stat-number" id="totalEntries">0</span>
                            <div class="stat-label">Total Entries</div>
                        </div>
                        <div class="stat-item">
                            <span class="stat-number" id="categoriesCount">0</span>
                            <div class="stat-label">Categories</div>
                        </div>
                        <div class="stat-item">
                            <span class="stat-number" id="latestEntry">-</span>
                            <div class="stat-label">Latest Entry</div>
                        </div>
                    </div>
                </div>

                <div class="loading" id="loading">Loading timeline...</div>
                <div class="error" id="error" style="display: none;"></div>
                <div class="timeline" id="timeline"></div>
            </div>

            <script>
                async function loadTimeline() {
                    try {
                        const response = await fetch('timeline-data.json');
                        if (!response.ok) {
                            throw new Error('Failed to load timeline data');
                        }

                        const entries = await response.json();
                        displayTimeline(entries);
                        displayStats(entries);

                    } catch (error) {
                        console.error('Error loading timeline:', error);
                        document.getElementById('loading').style.display = 'none';
                        document.getElementById('error').style.display = 'block';
                        document.getElementById('error').textContent = 'Failed to load timeline. Please try again later.';
                    }
                }

                function displayStats(entries) {
                    const categories = [...new Set(entries.map(entry => entry.category))];
                    const latest = entries.length > 0 ? new Date(entries[0].date).toLocaleDateString() : '-';

                    document.getElementById('totalEntries').textContent = entries.length;
                    document.getElementById('categoriesCount').textContent = categories.length;
                    document.getElementById('latestEntry').textContent = latest;
                    document.getElementById('stats').style.display = 'block';
                }

                function displayTimeline(entries) {
                    const timelineContainer = document.getElementById('timeline');
                    const loading = document.getElementById('loading');

                    loading.style.display = 'none';

                    if (entries.length === 0) {
                        timelineContainer.innerHTML = '<p style="text-align: center; color: white;">No entries found.</p>';
                        return;
                    }

                    const timelineHTML = entries.map(entry => {
                        const date = new Date(entry.date).toLocaleDateString('en-US', {
                            year: 'numeric',
                            month: 'short',
                            day: 'numeric'
                        });

                        return `
                            <div class="timeline-item">
                                <div class="timeline-header">
                                    <a href="${entry.url}" class="timeline-title" target="_blank" rel="noopener noreferrer">
                                        ${entry.title}
                                    </a>
                                    <div class="timeline-meta">
                                        <span class="timeline-category">${entry.category}</span>
                                        <span class="timeline-date">${date}</span>
                                    </div>
                                </div>
                                <div class="timeline-description">
                                    ${entry.description}
                                </div>
                            </div>
                        `;
                    }).join('');

                    timelineContainer.innerHTML = timelineHTML;
                }

                // Load timeline on page load
                loadTimeline();
            </script>
        </body>
        </html>
        EOF

    - name: Setup Pages
      uses: actions/configure-pages@v4

    - name: Upload artifact
      uses: actions/upload-pages-artifact@v3
      with:
        path: 'docs'

  deploy:
    environment:
      name: github-pages
      url: ${{ steps.deployment.outputs.page_url }}
    runs-on: ubuntu-latest
    needs: build-timeline
    if: github.ref == 'refs/heads/main'

    steps:
    - name: Deploy to GitHub Pages
      id: deployment
      uses: actions/deploy-pages@v4

I added it to my TIL repo and tested to see how it looked. It was a great start!

I then followed up with

this is a great start! however, it looks like the are a few issues: 1. All of the dates on the timeline are the published date and not the date of the actual learning. There is a SQLite database that stores this kind of meta data 2. The order of the items matches the order they are displayed on the Read Me, which is by category and not by date (see point 1) 3. the colors for each TIL seem to just rotate in order and aren't associated with any specific category of TIL

Claude went to work and generated a new GitHub Action for me

name: Build TIL Timeline

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]
  schedule:
    # Rebuild daily at 6 AM UTC
    - cron: '0 6 * * *'

permissions:
  contents: read
  pages: write
  id-token: write

jobs:
  build-timeline:
    runs-on: ubuntu-latest

    steps:
    - name: Check out repo
      uses: actions/checkout@v4
      with:
        fetch-depth: 0  # Need full history for git analysis

    - name: Set up Python
      uses: actions/setup-python@v5
      with:
        python-version: 3.12

    - uses: actions/cache@v4
      name: Configure pip caching
      with:
        path: ~/.cache/pip
        key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt') }}
        restore-keys: |
          ${{ runner.os }}-pip-

    - name: Install Python dependencies
      run: |
        python -m pip install --upgrade pip
        pip install -r requirements.txt

    - name: Build database
      run: python build_database.py

    - name: Generate timeline data from SQLite
      run: |
        python << 'EOF'
        import sqlite3
        import json
        import os
        from pathlib import Path

        # Connect to the SQLite database
        db_path = Path("tils.db")
        if not db_path.exists():
            print("Database not found!")
            exit(1)

        conn = sqlite3.connect(db_path)
        conn.row_factory = sqlite3.Row  # Enable dict-like access to rows

        # Query all TIL entries, ordered by created date (actual learning date)
        cursor = conn.execute("""
            SELECT
                path,
                slug,
                topic,
                title,
                url,
                body,
                created,
                created_utc,
                updated,
                updated_utc
            FROM til
            ORDER BY created_utc DESC
        """)

        entries = []
        for row in cursor:
            # Extract first paragraph as description, excluding headers and code blocks
            lines = row['body'].split('\n')
            description = ''
            for line in lines:
                line = line.strip()
                if line and not line.startswith('#') and not line.startswith('```') and line != '---':
                    description = line
                    break

            # Clean up the description
            if len(description) > 200:
                description = description[:200] + '...'

            # Extract date from created_utc (YYYY-MM-DD format)
            created_date = row['created_utc'].split('T')[0] if row['created_utc'] else row['created'].split('T')[0]

            entry = {
                'title': row['title'],
                'description': description or 'No description available',
                'date': created_date,
                'url': row['url'],
                'category': row['topic'],
                'fileName': row['path'].replace('_', '/'),  # Convert back from slug format
                'slug': row['slug']
            }
            entries.append(entry)

        conn.close()

        # Create output directory
        os.makedirs('docs', exist_ok=True)

        # Write timeline data
        with open('docs/timeline-data.json', 'w') as f:
            json.dump(entries, f, indent=2)

        print(f"Generated timeline with {len(entries)} entries")

        # Generate category statistics for consistent coloring
        categories = {}
        for entry in entries:
            cat = entry['category']
            if cat not in categories:
                categories[cat] = len([e for e in entries if e['category'] == cat])

        # Sort categories by count (most popular first) for consistent color assignment
        sorted_categories = sorted(categories.items(), key=lambda x: x[1], reverse=True)
        category_colors = {}

        # Define a set of distinct colors for categories
        color_palette = [
            '#4CAF50',  # Green
            '#2196F3',  # Blue
            '#FF9800',  # Orange
            '#E91E63',  # Pink
            '#9C27B0',  # Purple
            '#00BCD4',  # Cyan
            '#FF5722',  # Deep Orange
            '#795548',  # Brown
            '#607D8B',  # Blue Grey
            '#FFC107',  # Amber
            '#8BC34A',  # Light Green
            '#3F51B5',  # Indigo
            '#F44336',  # Red
            '#009688',  # Teal
            '#CDDC39',  # Lime
        ]

        for i, (category, count) in enumerate(sorted_categories):
            category_colors[category] = color_palette[i % len(color_palette)]

        # Write category color mapping
        with open('docs/category-colors.json', 'w') as f:
            json.dump(category_colors, f, indent=2)

        print(f"Generated color mapping for {len(category_colors)} categories")
        EOF

    - name: Generate timeline HTML
      run: |
        cat > docs/index.html << 'EOF'
        <!DOCTYPE html>
        <html lang="en">
        <head>
            <meta charset="UTF-8">
            <meta name="viewport" content="width=device-width, initial-scale=1.0">
            <title>Ryan's Today I Learned Timeline</title>
            <meta name="description" content="A chronological timeline of learning discoveries from software development, featuring insights on Python, Django, SQL, and more.">
            <style>
                * {
                    margin: 0;
                    padding: 0;
                    box-sizing: border-box;
                }

                body {
                    font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', system-ui, sans-serif;
                    background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
                    min-height: 100vh;
                    color: #333;
                }

                .container {
                    max-width: 1200px;
                    margin: 0 auto;
                    padding: 2rem;
                }

                .header {
                    text-align: center;
                    margin-bottom: 3rem;
                    color: white;
                }

                .header h1 {
                    font-size: 3rem;
                    margin-bottom: 1rem;
                    text-shadow: 2px 2px 4px rgba(0,0,0,0.3);
                }

                .header p {
                    font-size: 1.2rem;
                    opacity: 0.9;
                    max-width: 600px;
                    margin: 0 auto;
                }

                .filters {
                    background: rgba(255,255,255,0.95);
                    border-radius: 12px;
                    padding: 1.5rem;
                    margin-bottom: 2rem;
                    backdrop-filter: blur(10px);
                    border: 1px solid rgba(255,255,255,0.2);
                }

                .filter-group {
                    display: flex;
                    flex-wrap: wrap;
                    gap: 0.5rem;
                    align-items: center;
                }

                .filter-label {
                    font-weight: 600;
                    margin-right: 1rem;
                    color: #666;
                }

                .category-filter {
                    padding: 0.4rem 0.8rem;
                    border-radius: 20px;
                    border: 2px solid transparent;
                    background: #f8f9fa;
                    color: #666;
                    cursor: pointer;
                    transition: all 0.3s ease;
                    font-size: 0.9rem;
                    user-select: none;
                }

                .category-filter:hover {
                    transform: translateY(-2px);
                    box-shadow: 0 4px 8px rgba(0,0,0,0.1);
                }

                .category-filter.active {
                    color: white;
                    border-color: currentColor;
                    font-weight: 600;
                }

                .timeline {
                    position: relative;
                    margin-top: 2rem;
                }

                .timeline::before {
                    content: '';
                    position: absolute;
                    left: 2rem;
                    top: 0;
                    bottom: 0;
                    width: 2px;
                    background: linear-gradient(to bottom, #4CAF50, #2196F3, #FF9800, #E91E63);
                }

                .timeline-item {
                    position: relative;
                    margin-bottom: 2rem;
                    margin-left: 4rem;
                    background: white;
                    border-radius: 12px;
                    padding: 1.5rem;
                    box-shadow: 0 8px 25px rgba(0,0,0,0.1);
                    transition: all 0.3s ease;
                    opacity: 1;
                }

                .timeline-item.hidden {
                    display: none;
                }

                .timeline-item:hover {
                    transform: translateY(-5px);
                    box-shadow: 0 15px 35px rgba(0,0,0,0.15);
                }

                .timeline-item::before {
                    content: '';
                    position: absolute;
                    left: -3rem;
                    top: 2rem;
                    width: 16px;
                    height: 16px;
                    background: var(--category-color, #4CAF50);
                    border: 3px solid white;
                    border-radius: 50%;
                    box-shadow: 0 0 0 3px rgba(76, 175, 80, 0.3);
                }

                .timeline-header {
                    display: flex;
                    justify-content: space-between;
                    align-items: flex-start;
                    margin-bottom: 1rem;
                    flex-wrap: wrap;
                    gap: 1rem;
                }

                .timeline-title {
                    font-size: 1.4rem;
                    font-weight: 600;
                    color: #2c3e50;
                    text-decoration: none;
                    flex-grow: 1;
                    transition: color 0.3s ease;
                }

                .timeline-title:hover {
                    color: #3498db;
                }

                .timeline-meta {
                    display: flex;
                    gap: 1rem;
                    align-items: center;
                    flex-shrink: 0;
                }

                .timeline-date {
                    background: linear-gradient(135deg, #667eea, #764ba2);
                    color: white;
                    padding: 0.5rem 1rem;
                    border-radius: 20px;
                    font-size: 0.9rem;
                    font-weight: 500;
                }

                .timeline-category {
                    background: var(--category-color, #f8f9fa);
                    color: white;
                    padding: 0.4rem 0.8rem;
                    border-radius: 15px;
                    font-size: 0.8rem;
                    font-weight: 500;
                    border: 1px solid rgba(255,255,255,0.2);
                }

                .timeline-description {
                    color: #5a6c7d;
                    line-height: 1.6;
                    font-size: 1rem;
                }

                .loading {
                    text-align: center;
                    padding: 3rem;
                    color: white;
                    font-size: 1.2rem;
                }

                .error {
                    background: #f8d7da;
                    color: #721c24;
                    padding: 1rem;
                    border-radius: 8px;
                    border: 1px solid #f5c6cb;
                    margin-bottom: 2rem;
                }

                .stats {
                    background: rgba(255,255,255,0.95);
                    border-radius: 12px;
                    padding: 1.5rem;
                    margin-bottom: 2rem;
                    backdrop-filter: blur(10px);
                    border: 1px solid rgba(255,255,255,0.2);
                }

                .stats-grid {
                    display: grid;
                    grid-template-columns: repeat(auto-fit, minmax(200px, 1fr));
                    gap: 1rem;
                    text-align: center;
                }

                .stat-item {
                    padding: 1rem;
                }

                .stat-number {
                    font-size: 2rem;
                    font-weight: bold;
                    color: #667eea;
                    display: block;
                }

                .stat-label {
                    color: #666;
                    font-size: 0.9rem;
                    margin-top: 0.5rem;
                }

                @media (max-width: 768px) {
                    .container {
                        padding: 1rem;
                    }

                    .header h1 {
                        font-size: 2rem;
                    }

                    .timeline::before {
                        left: 1rem;
                    }

                    .timeline-item {
                        margin-left: 2rem;
                        padding: 1rem;
                    }

                    .timeline-item::before {
                        left: -2rem;
                    }

                    .timeline-header {
                        flex-direction: column;
                        align-items: stretch;
                    }

                    .timeline-meta {
                        justify-content: space-between;
                    }

                    .filter-group {
                        flex-direction: column;
                        align-items: stretch;
                        gap: 1rem;
                    }

                    .category-filter {
                        text-align: center;
                    }
                }
            </style>
        </head>
        <body>
            <div class="container">
                <div class="header">
                    <h1>Today I Learned</h1>
                    <p>A chronological timeline of discoveries, learnings, and insights from my development journey</p>
                </div>

                <div class="stats" id="stats" style="display: none;">
                    <div class="stats-grid">
                        <div class="stat-item">
                            <span class="stat-number" id="totalEntries">0</span>
                            <div class="stat-label">Total Entries</div>
                        </div>
                        <div class="stat-item">
                            <span class="stat-number" id="categoriesCount">0</span>
                            <div class="stat-label">Categories</div>
                        </div>
                        <div class="stat-item">
                            <span class="stat-number" id="latestEntry">-</span>
                            <div class="stat-label">Latest Entry</div>
                        </div>
                        <div class="stat-item">
                            <span class="stat-number" id="filteredCount">0</span>
                            <div class="stat-label">Showing</div>
                        </div>
                    </div>
                </div>

                <div class="filters" id="filters" style="display: none;">
                    <div class="filter-group">
                        <span class="filter-label">Filter by category:</span>
                        <div id="categoryFilters"></div>
                    </div>
                </div>

                <div class="loading" id="loading">Loading timeline...</div>
                <div class="error" id="error" style="display: none;"></div>
                <div class="timeline" id="timeline"></div>
            </div>

            <script>
                let allEntries = [];
                let categoryColors = {};
                let activeCategory = null;

                async function loadTimeline() {
                    try {
                        // Load timeline data and category colors
                        const [entriesResponse, colorsResponse] = await Promise.all([
                            fetch('timeline-data.json'),
                            fetch('category-colors.json')
                        ]);

                        if (!entriesResponse.ok || !colorsResponse.ok) {
                            throw new Error('Failed to load timeline data');
                        }

                        allEntries = await entriesResponse.json();
                        categoryColors = await colorsResponse.json();

                        displayTimeline(allEntries);
                        displayStats(allEntries);
                        createCategoryFilters();

                    } catch (error) {
                        console.error('Error loading timeline:', error);
                        document.getElementById('loading').style.display = 'none';
                        document.getElementById('error').style.display = 'block';
                        document.getElementById('error').textContent = 'Failed to load timeline. Please try again later.';
                    }
                }

                function createCategoryFilters() {
                    const categories = [...new Set(allEntries.map(entry => entry.category))];
                    const filtersContainer = document.getElementById('categoryFilters');

                    // Add "All" filter
                    const allFilter = document.createElement('span');
                    allFilter.className = 'category-filter active';
                    allFilter.textContent = 'All';
                    allFilter.onclick = () => filterByCategory(null);
                    filtersContainer.appendChild(allFilter);

                    // Add category filters
                    categories.sort().forEach(category => {
                        const filter = document.createElement('span');
                        filter.className = 'category-filter';
                        filter.textContent = category;
                        filter.style.setProperty('--category-color', categoryColors[category] || '#666');
                        filter.onclick = () => filterByCategory(category);
                        filtersContainer.appendChild(filter);
                    });

                    document.getElementById('filters').style.display = 'block';
                }

                function filterByCategory(category) {
                    activeCategory = category;

                    // Update filter button states
                    document.querySelectorAll('.category-filter').forEach(filter => {
                        filter.classList.remove('active');
                        if ((category === null && filter.textContent === 'All') ||
                            filter.textContent === category) {
                            filter.classList.add('active');
                            if (category !== null) {
                                filter.style.background = categoryColors[category];
                            }
                        }
                    });

                    // Filter timeline items
                    const filteredEntries = category ?
                        allEntries.filter(entry => entry.category === category) :
                        allEntries;

                    displayTimeline(filteredEntries);
                    updateFilteredCount(filteredEntries.length);
                }

                function updateFilteredCount(count) {
                    document.getElementById('filteredCount').textContent = count;
                }

                function displayStats(entries) {
                    const categories = [...new Set(entries.map(entry => entry.category))];
                    const latest = entries.length > 0 ? new Date(entries[0].date).toLocaleDateString() : '-';

                    document.getElementById('totalEntries').textContent = entries.length;
                    document.getElementById('categoriesCount').textContent = categories.length;
                    document.getElementById('latestEntry').textContent = latest;
                    document.getElementById('filteredCount').textContent = entries.length;
                    document.getElementById('stats').style.display = 'block';
                }

                function displayTimeline(entries) {
                    const timelineContainer = document.getElementById('timeline');
                    const loading = document.getElementById('loading');

                    loading.style.display = 'none';

                    if (entries.length === 0) {
                        timelineContainer.innerHTML = '<p style="text-align: center; color: white;">No entries found.</p>';
                        return;
                    }

                    const timelineHTML = entries.map(entry => {
                        const date = new Date(entry.date).toLocaleDateString('en-US', {
                            year: 'numeric',
                            month: 'short',
                            day: 'numeric'
                        });

                        const categoryColor = categoryColors[entry.category] || '#666';

                        return `
                            <div class="timeline-item" style="--category-color: ${categoryColor}">
                                <div class="timeline-header">
                                    <a href="${entry.url}" class="timeline-title" target="_blank" rel="noopener noreferrer">
                                        ${entry.title}
                                    </a>
                                    <div class="timeline-meta">
                                        <span class="timeline-category" style="--category-color: ${categoryColor}; background: ${categoryColor};">
                                            ${entry.category}
                                        </span>
                                        <span class="timeline-date">${date}</span>
                                    </div>
                                </div>
                                <div class="timeline-description">
                                    ${entry.description}
                                </div>
                            </div>
                        `;
                    }).join('');

                    timelineContainer.innerHTML = timelineHTML;
                }

                // Load timeline on page load
                loadTimeline();
            </script>
        </body>
        </html>
        EOF

    - name: Setup Pages
      uses: actions/configure-pages@v4

    - name: Upload artifact
      uses: actions/upload-pages-artifact@v3
      with:
        path: 'docs'

  deploy:
    environment:
      name: github-pages
      url: ${{ steps.deployment.outputs.page_url }}
    runs-on: ubuntu-latest
    needs: build-timeline
    if: github.ref == 'refs/heads/main'

    steps:
    - name: Deploy to GitHub Pages
      id: deployment
      uses: actions/deploy-pages@v4

Another huge step forward! However, there was one last change that was needed

oh wow! much better! The last issue I'm seeing is that because I have so many categories, the Filter by category needs a way to wrap based on the page width. Right now it's on a single line, but would likely benefit from a css style that will wrap the lines

And I got this

name: Build TIL Timeline

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]
  schedule:
    # Rebuild daily at 6 AM UTC
    - cron: '0 6 * * *'

permissions:
  contents: read
  pages: write
  id-token: write

jobs:
  build-timeline:
    runs-on: ubuntu-latest

    steps:
    - name: Check out repo
      uses: actions/checkout@v4
      with:
        fetch-depth: 0  # Need full history for git analysis

    - name: Set up Python
      uses: actions/setup-python@v5
      with:
        python-version: 3.12

    - uses: actions/cache@v4
      name: Configure pip caching
      with:
        path: ~/.cache/pip
        key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt') }}
        restore-keys: |
          ${{ runner.os }}-pip-

    - name: Install Python dependencies
      run: |
        python -m pip install --upgrade pip
        pip install -r requirements.txt

    - name: Build database
      run: python build_database.py

    - name: Generate timeline data from SQLite
      run: |
        python << 'EOF'
        import sqlite3
        import json
        import os
        from pathlib import Path

        # Connect to the SQLite database
        db_path = Path("tils.db")
        if not db_path.exists():
            print("Database not found!")
            exit(1)

        conn = sqlite3.connect(db_path)
        conn.row_factory = sqlite3.Row  # Enable dict-like access to rows

        # Query all TIL entries, ordered by created date (actual learning date)
        cursor = conn.execute("""
            SELECT
                path,
                slug,
                topic,
                title,
                url,
                body,
                created,
                created_utc,
                updated,
                updated_utc
            FROM til
            ORDER BY created_utc DESC
        """)

        entries = []
        for row in cursor:
            # Extract first paragraph as description, excluding headers and code blocks
            lines = row['body'].split('\n')
            description = ''
            for line in lines:
                line = line.strip()
                if line and not line.startswith('#') and not line.startswith('```') and line != '---':
                    description = line
                    break

            # Clean up the description
            if len(description) > 200:
                description = description[:200] + '...'

            # Extract date from created_utc (YYYY-MM-DD format)
            created_date = row['created_utc'].split('T')[0] if row['created_utc'] else row['created'].split('T')[0]

            entry = {
                'title': row['title'],
                'description': description or 'No description available',
                'date': created_date,
                'url': row['url'],
                'category': row['topic'],
                'fileName': row['path'].replace('_', '/'),  # Convert back from slug format
                'slug': row['slug']
            }
            entries.append(entry)

        conn.close()

        # Create output directory
        os.makedirs('docs', exist_ok=True)

        # Write timeline data
        with open('docs/timeline-data.json', 'w') as f:
            json.dump(entries, f, indent=2)

        print(f"Generated timeline with {len(entries)} entries")

        # Generate category statistics for consistent coloring
        categories = {}
        for entry in entries:
            cat = entry['category']
            if cat not in categories:
                categories[cat] = len([e for e in entries if e['category'] == cat])

        # Sort categories by count (most popular first) for consistent color assignment
        sorted_categories = sorted(categories.items(), key=lambda x: x[1], reverse=True)
        category_colors = {}

        # Define a set of distinct colors for categories
        color_palette = [
            '#4CAF50',  # Green
            '#2196F3',  # Blue
            '#FF9800',  # Orange
            '#E91E63',  # Pink
            '#9C27B0',  # Purple
            '#00BCD4',  # Cyan
            '#FF5722',  # Deep Orange
            '#795548',  # Brown
            '#607D8B',  # Blue Grey
            '#FFC107',  # Amber
            '#8BC34A',  # Light Green
            '#3F51B5',  # Indigo
            '#F44336',  # Red
            '#009688',  # Teal
            '#CDDC39',  # Lime
        ]

        for i, (category, count) in enumerate(sorted_categories):
            category_colors[category] = color_palette[i % len(color_palette)]

        # Write category color mapping
        with open('docs/category-colors.json', 'w') as f:
            json.dump(category_colors, f, indent=2)

        print(f"Generated color mapping for {len(category_colors)} categories")
        EOF

    - name: Generate timeline HTML
      run: |
        cat > docs/index.html << 'EOF'
        <!DOCTYPE html>
        <html lang="en">
        <head>
            <meta charset="UTF-8">
            <meta name="viewport" content="width=device-width, initial-scale=1.0">
            <title>Ryan's Today I Learned Timeline</title>
            <meta name="description" content="A chronological timeline of learning discoveries from software development, featuring insights on Python, Django, SQL, and more.">
            <style>
                * {
                    margin: 0;
                    padding: 0;
                    box-sizing: border-box;
                }

                body {
                    font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', system-ui, sans-serif;
                    background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
                    min-height: 100vh;
                    color: #333;
                }

                .container {
                    max-width: 1200px;
                    margin: 0 auto;
                    padding: 2rem;
                }

                .header {
                    text-align: center;
                    margin-bottom: 3rem;
                    color: white;
                }

                .header h1 {
                    font-size: 3rem;
                    margin-bottom: 1rem;
                    text-shadow: 2px 2px 4px rgba(0,0,0,0.3);
                }

                .header p {
                    font-size: 1.2rem;
                    opacity: 0.9;
                    max-width: 600px;
                    margin: 0 auto;
                }

                .filters {
                    background: rgba(255,255,255,0.95);
                    border-radius: 12px;
                    padding: 1.5rem;
                    margin-bottom: 2rem;
                    backdrop-filter: blur(10px);
                    border: 1px solid rgba(255,255,255,0.2);
                }

                .filter-group {
                    display: flex;
                    flex-direction: column;
                    gap: 1rem;
                }

                .filter-label {
                    font-weight: 600;
                    color: #666;
                    margin-bottom: 0.5rem;
                }

                .category-filters-container {
                    display: flex;
                    flex-wrap: wrap;
                    gap: 0.5rem;
                    align-items: center;
                }

                .category-filter {
                    padding: 0.4rem 0.8rem;
                    border-radius: 20px;
                    border: 2px solid transparent;
                    background: #f8f9fa;
                    color: #666;
                    cursor: pointer;
                    transition: all 0.3s ease;
                    font-size: 0.9rem;
                    user-select: none;
                }

                .category-filter:hover {
                    transform: translateY(-2px);
                    box-shadow: 0 4px 8px rgba(0,0,0,0.1);
                }

                .category-filter.active {
                    color: white;
                    border-color: currentColor;
                    font-weight: 600;
                }

                .timeline {
                    position: relative;
                    margin-top: 2rem;
                }

                .timeline::before {
                    content: '';
                    position: absolute;
                    left: 2rem;
                    top: 0;
                    bottom: 0;
                    width: 2px;
                    background: linear-gradient(to bottom, #4CAF50, #2196F3, #FF9800, #E91E63);
                }

                .timeline-item {
                    position: relative;
                    margin-bottom: 2rem;
                    margin-left: 4rem;
                    background: white;
                    border-radius: 12px;
                    padding: 1.5rem;
                    box-shadow: 0 8px 25px rgba(0,0,0,0.1);
                    transition: all 0.3s ease;
                    opacity: 1;
                }

                .timeline-item.hidden {
                    display: none;
                }

                .timeline-item:hover {
                    transform: translateY(-5px);
                    box-shadow: 0 15px 35px rgba(0,0,0,0.15);
                }

                .timeline-item::before {
                    content: '';
                    position: absolute;
                    left: -3rem;
                    top: 2rem;
                    width: 16px;
                    height: 16px;
                    background: var(--category-color, #4CAF50);
                    border: 3px solid white;
                    border-radius: 50%;
                    box-shadow: 0 0 0 3px rgba(76, 175, 80, 0.3);
                }

                .timeline-header {
                    display: flex;
                    justify-content: space-between;
                    align-items: flex-start;
                    margin-bottom: 1rem;
                    flex-wrap: wrap;
                    gap: 1rem;
                }

                .timeline-title {
                    font-size: 1.4rem;
                    font-weight: 600;
                    color: #2c3e50;
                    text-decoration: none;
                    flex-grow: 1;
                    transition: color 0.3s ease;
                }

                .timeline-title:hover {
                    color: #3498db;
                }

                .timeline-meta {
                    display: flex;
                    gap: 1rem;
                    align-items: center;
                    flex-shrink: 0;
                }

                .timeline-date {
                    background: linear-gradient(135deg, #667eea, #764ba2);
                    color: white;
                    padding: 0.5rem 1rem;
                    border-radius: 20px;
                    font-size: 0.9rem;
                    font-weight: 500;
                }

                .timeline-category {
                    background: var(--category-color, #f8f9fa);
                    color: white;
                    padding: 0.4rem 0.8rem;
                    border-radius: 15px;
                    font-size: 0.8rem;
                    font-weight: 500;
                    border: 1px solid rgba(255,255,255,0.2);
                }

                .timeline-description {
                    color: #5a6c7d;
                    line-height: 1.6;
                    font-size: 1rem;
                }

                .loading {
                    text-align: center;
                    padding: 3rem;
                    color: white;
                    font-size: 1.2rem;
                }

                .error {
                    background: #f8d7da;
                    color: #721c24;
                    padding: 1rem;
                    border-radius: 8px;
                    border: 1px solid #f5c6cb;
                    margin-bottom: 2rem;
                }

                .stats {
                    background: rgba(255,255,255,0.95);
                    border-radius: 12px;
                    padding: 1.5rem;
                    margin-bottom: 2rem;
                    backdrop-filter: blur(10px);
                    border: 1px solid rgba(255,255,255,0.2);
                }

                .stats-grid {
                    display: grid;
                    grid-template-columns: repeat(auto-fit, minmax(200px, 1fr));
                    gap: 1rem;
                    text-align: center;
                }

                .stat-item {
                    padding: 1rem;
                }

                .stat-number {
                    font-size: 2rem;
                    font-weight: bold;
                    color: #667eea;
                    display: block;
                }

                .stat-label {
                    color: #666;
                    font-size: 0.9rem;
                    margin-top: 0.5rem;
                }

                @media (max-width: 768px) {
                    .container {
                        padding: 1rem;
                    }

                    .header h1 {
                        font-size: 2rem;
                    }

                    .timeline::before {
                        left: 1rem;
                    }

                    .timeline-item {
                        margin-left: 2rem;
                        padding: 1rem;
                    }

                    .timeline-item::before {
                        left: -2rem;
                    }

                    .timeline-header {
                        flex-direction: column;
                        align-items: stretch;
                    }

                    .timeline-meta {
                        justify-content: space-between;
                    }

                    .filter-group {
                        gap: 1rem;
                    }

                    .category-filters-container {
                        justify-content: center;
                    }
                }
            </style>
        </head>
        <body>
            <div class="container">
                <div class="header">
                    <h1>Today I Learned</h1>
                    <p>A chronological timeline of discoveries, learnings, and insights from my development journey</p>
                </div>

                <div class="stats" id="stats" style="display: none;">
                    <div class="stats-grid">
                        <div class="stat-item">
                            <span class="stat-number" id="totalEntries">0</span>
                            <div class="stat-label">Total Entries</div>
                        </div>
                        <div class="stat-item">
                            <span class="stat-number" id="categoriesCount">0</span>
                            <div class="stat-label">Categories</div>
                        </div>
                        <div class="stat-item">
                            <span class="stat-number" id="latestEntry">-</span>
                            <div class="stat-label">Latest Entry</div>
                        </div>
                        <div class="stat-item">
                            <span class="stat-number" id="filteredCount">0</span>
                            <div class="stat-label">Showing</div>
                        </div>
                    </div>
                </div>

                <div class="filters" id="filters" style="display: none;">
                    <div class="filter-group">
                        <span class="filter-label">Filter by category:</span>
                        <div class="category-filters-container" id="categoryFilters"></div>
                    </div>
                </div>

                <div class="loading" id="loading">Loading timeline...</div>
                <div class="error" id="error" style="display: none;"></div>
                <div class="timeline" id="timeline"></div>
            </div>

            <script>
                let allEntries = [];
                let categoryColors = {};
                let activeCategory = null;

                async function loadTimeline() {
                    try {
                        // Load timeline data and category colors
                        const [entriesResponse, colorsResponse] = await Promise.all([
                            fetch('timeline-data.json'),
                            fetch('category-colors.json')
                        ]);

                        if (!entriesResponse.ok || !colorsResponse.ok) {
                            throw new Error('Failed to load timeline data');
                        }

                        allEntries = await entriesResponse.json();
                        categoryColors = await colorsResponse.json();

                        displayTimeline(allEntries);
                        displayStats(allEntries);
                        createCategoryFilters();

                    } catch (error) {
                        console.error('Error loading timeline:', error);
                        document.getElementById('loading').style.display = 'none';
                        document.getElementById('error').style.display = 'block';
                        document.getElementById('error').textContent = 'Failed to load timeline. Please try again later.';
                    }
                }

                function createCategoryFilters() {
                    const categories = [...new Set(allEntries.map(entry => entry.category))];
                    const filtersContainer = document.getElementById('categoryFilters');

                    // Add "All" filter
                    const allFilter = document.createElement('span');
                    allFilter.className = 'category-filter active';
                    allFilter.textContent = 'All';
                    allFilter.onclick = () => filterByCategory(null);
                    filtersContainer.appendChild(allFilter);

                    // Add category filters
                    categories.sort().forEach(category => {
                        const filter = document.createElement('span');
                        filter.className = 'category-filter';
                        filter.textContent = category;
                        filter.style.setProperty('--category-color', categoryColors[category] || '#666');
                        filter.onclick = () => filterByCategory(category);
                        filtersContainer.appendChild(filter);
                    });

                    document.getElementById('filters').style.display = 'block';
                }

                function filterByCategory(category) {
                    activeCategory = category;

                    // Update filter button states
                    document.querySelectorAll('.category-filter').forEach(filter => {
                        filter.classList.remove('active');
                        if ((category === null && filter.textContent === 'All') ||
                            filter.textContent === category) {
                            filter.classList.add('active');
                            if (category !== null) {
                                filter.style.background = categoryColors[category];
                            }
                        }
                    });

                    // Filter timeline items
                    const filteredEntries = category ?
                        allEntries.filter(entry => entry.category === category) :
                        allEntries;

                    displayTimeline(filteredEntries);
                    updateFilteredCount(filteredEntries.length);
                }

                function updateFilteredCount(count) {
                    document.getElementById('filteredCount').textContent = count;
                }

                function displayStats(entries) {
                    const categories = [...new Set(entries.map(entry => entry.category))];
                    const latest = entries.length > 0 ? new Date(entries[0].date).toLocaleDateString() : '-';

                    document.getElementById('totalEntries').textContent = entries.length;
                    document.getElementById('categoriesCount').textContent = categories.length;
                    document.getElementById('latestEntry').textContent = latest;
                    document.getElementById('filteredCount').textContent = entries.length;
                    document.getElementById('stats').style.display = 'block';
                }

                function displayTimeline(entries) {
                    const timelineContainer = document.getElementById('timeline');
                    const loading = document.getElementById('loading');

                    loading.style.display = 'none';

                    if (entries.length === 0) {
                        timelineContainer.innerHTML = '<p style="text-align: center; color: white;">No entries found.</p>';
                        return;
                    }

                    const timelineHTML = entries.map(entry => {
                        const date = new Date(entry.date).toLocaleDateString('en-US', {
                            year: 'numeric',
                            month: 'short',
                            day: 'numeric'
                        });

                        const categoryColor = categoryColors[entry.category] || '#666';

                        return `
                            <div class="timeline-item" style="--category-color: ${categoryColor}">
                                <div class="timeline-header">
                                    <a href="${entry.url}" class="timeline-title" target="_blank" rel="noopener noreferrer">
                                        ${entry.title}
                                    </a>
                                    <div class="timeline-meta">
                                        <span class="timeline-category" style="--category-color: ${categoryColor}; background: ${categoryColor};">
                                            ${entry.category}
                                        </span>
                                        <span class="timeline-date">${date}</span>
                                    </div>
                                </div>
                                <div class="timeline-description">
                                    ${entry.description}
                                </div>
                            </div>
                        `;
                    }).join('');

                    timelineContainer.innerHTML = timelineHTML;
                }

                // Load timeline on page load
                loadTimeline();
            </script>
        </body>
        </html>
        EOF

    - name: Setup Pages
      uses: actions/configure-pages@v4

    - name: Upload artifact
      uses: actions/upload-pages-artifact@v3
      with:
        path: 'docs'

  deploy:
    environment:
      name: github-pages
      url: ${{ steps.deployment.outputs.page_url }}
    runs-on: ubuntu-latest
    needs: build-timeline
    if: github.ref == 'refs/heads/main'

    steps:
    - name: Deploy to GitHub Pages
      id: deployment
      uses: actions/deploy-pages@v4

All told from start to finish, this took about 20 minutes. There are still some potential updates, but for 20 minutes of work I was able to take a 'wild' idea that I would have never been able to do before and had something that I'm actually excited about! And it has the added bonus of encouraging me to write more TILs because I now have this nice looking timeline of my TILs.

Fun with MCPs

2025-06-02T00:00:00-07:00

Special Thanks to Jeff Triplett who provided an example that really got me started on better understanding of how this all works.

In trying to wrap my head around MCPs over the long Memorial weekend I had a breakthrough. I'm not really sure why this was so hard for me to grok, but now something seems to have clicked.

I am working with Pydantic AI and so I'll be using that as an example, but since MCPs are a standard protocol, these concepts apply broadly across different implementations.

What is Model Context Protocol (MCP)?

Per the Anthropic announcement (from November 2024!!!!)

The Model Context Protocol is an open standard that enables developers to build secure, two-way connections between their data sources and AI-powered tools. The architecture is straightforward: developers can either expose their data through MCP servers or build AI applications (MCP clients) that connect to these servers.

What this means is that there is a standard way to extend models like Claude, or OpenAI to include other information. That information can be files on the file system, data in a database, etc.

(Potential) Real World Example

I work for a Healthcare organization in Southern California. One of the biggest challenges with onboarding new hires (and honestly can be a challenge for people that have been with the organization for a long time) is who to reach out to for support on which specific application.

Typically a user will send an email to one of the support teams, and the email request can get bounced around for a while until it finally lands on the 'right' support desk. There's the potential to have the applications themselves include who to contact, but some applications are vendor supplied and there isn't always a way to do that.

Even if there were, in my experience those are often not noticed by users OR the users will think that the support email is for non-technical issues, like "Please update the phone number for this patient" and not issues like, "The web page isn't returning any results for me, but it is for my coworker."

Enter an MCP with a Local LLM

Let's say you have a service that allows you to search through a file system in a predefined set of directories. This service is run with the following command

npx -y --no-cache @modelcontextprotocol/server-filesystem /path/to/your/files

In Pydantic AI the use of the MCPServerStdio is using this same syntax only it breaks it into two parts

command
args

The command is any application in your $PATH like uvx or docker or npx, or you can explicitly define where the executable is by calling out its path, like /Users/ryancheley/.local/share/mise/installs/bun/latest/bin/bunx

The args are the commands you'd pass to your application.

Taking the command from above and breaking it down we can set up our MCP using the following

MCPServerStdio(
    "npx",
    args=[
        "-y",
        "--no-cache",
        "@modelcontextprotocol/server-filesystem",
        "/path/to/your/files",
    ]

Application of MCP with the Example

Since I work in Healthcare, and I want to be mindful of the protection of patient data, even if that data won't be exposed to this LLM, I'll use ollama to construct my example.

I created a support.csv file that contains the following information

Common Name of the Application
URL of the Application
Support Email
Support Extension
Department

I used the following prompt

Review the file support.csv and help me determine who I contact about questions related to CarePath Analytics.

Here are the contents of the support.csv file

Name	URL	Support Email	Support Extension	Department
MedFlow Solutions	https://medflow.com	support@medflow.com	1234	Clinical Systems
HealthTech Portal	https://healthtech-portal.org	help@medflow.com	3456	Patient Services
CarePath Analytics	https://carepath.io	support@medflow.com	4567	Data Analytics
VitalSign Monitor	https://vitalsign.net	support@medflow.com	1234	Clinical Systems
Patient Connect Hub	https://patientconnect.com	contact@medflow.com	3456	Patient Services
EHR Bridge	https://ehrbridge.org	support@medflow.com	2341	Integration Services
Clinical Workflow Pro	https://clinicalwf.com	support@medflow.com	1234	Clinical Systems
HealthData Sync	https://healthdata-sync.net	sync@medflow.com	6789	Integration Services
TeleHealth Connect	https://telehealth-connect.com	help@medflow.com	3456	Patient Services
MedRecord Central	https://medrecord.central	records@medflow.com	5678	Medical Records

The script is below:

# /// script
# requires-python = ">=3.12"
# dependencies = [
#     "pydantic-ai",
# ]
# ///

import asyncio
from pydantic_ai import Agent
from pydantic_ai.mcp import MCPServerStdio
from pydantic_ai.models.openai import OpenAIModel
from pydantic_ai.providers.openai import OpenAIProvider

async def main():
    # Configure the Ollama model using OpenAI-compatible API
    model = OpenAIModel(
        model_name='qwen3:8b',  # or whatever model you have installed locally
        provider=OpenAIProvider(base_url='http://localhost:11434/v1')
    )

    # Set up the MCP server to access our support files
    support_files_server = MCPServerStdio(
        "npx",
        args=[
            "-y",
            "@modelcontextprotocol/server-filesystem",
            "/path/to/your/files"  # Directory containing support.csv
        ]
    )

    # Create the agent with the model and MCP server
    agent = Agent(
        model=model,
        mcp_servers=[support_files_server],
    )

    # Run the agent with the MCP server
    async with agent.run_mcp_servers():
        # Get response from Ollama about support contact
        result = await agent.run(
            "Review the file `support.csv` and help me determine who I contact about questions related to CarePath Analytics?"        )
        print(result.output)

if __name__ == "__main__":
    asyncio.run(main())

As a user, if I ask, who do I contact about questions related to CarePath Analytics the LLM will search through the support.csv file and supply the email contact.

This example shows a command line script, and a Web Interface would probably be better for most users. That would be the next thing I'd try to do here.

Once that was done you could extend it to also include an MCP to write an email on the user's behalf. It could even ask probing questions to help make sure that the email had more context for the support team.

Some support systems have their own ticketing / issue tracking systems and it would be really valuable if this ticket could be written directly to that system. With the MCP this is possible.

We'd need to update the support.csv file with some information about direct writes via an API, and we'd need to secure the crap out of this, but it is possible.

Now, the user can be more confident that their issue will go to the team that it needs to and that their question / issue can be resolved much more quickly.

Firebirds 2024-25 Season

2025-05-21T00:00:00-07:00

The 2024-25 season for the Coachella Valley Firebirds ended on May 9th with a 2-0 loss to the Abbotsford Canucks. Overall, that series saw the Firebirds score

one goal in Game 1
one goal in Game 2
five goals in Game 3
no goals in Game 4

This isn't surprising given exactly how young the Firebirds were this season, but it was disappointing.

Coach Laxdal talked a lot about how young the team was and how on any given night we would have anywhere from seven to nine rookies that were in the starting lineup. And in a team of 24, that's a pretty big portion of guys out there who are very young.

That being said the disappointment is palpable the this is the earliest that the Firebirds have ever exited the postseason. Granted this is only their third year but we are typically used to seeing hockey for another seven weeks. When put into that perspective, it is really disappointing.

Still, I think there were some really bright spots from this year, including Leyton Roed, Jani Nyman, Nikke Kokko, Ryan Winterton, and Ty Nelson.

At the start of the season, I did indicate to a friend of mine (who also has season tickets) that I had pretty low expectations for the Firebirds and may have even indicated I wasn't sure that they would make the playoffs. The Pacific Division has 10 teams and 7 of them make the playoffs. I may have been a bit too pesimisitic in that analysis.

During the first round the Firebirds swept the Wrangerls 2-0. This is great, but they did manage to blow a 3-0 lead in game 1. The were able to win that game, but it took two plus Overtime periods (it ended a few minutes into the third OT).

Game two of that series did see the Firebirds win 2-0 with Nikke Kokko getting his first professional AHL shutout, which was great . But it's also a bummer that it took until the 74th game of the season for him to get his first shutout of the season. ¹

In six games, the Firebirds were 3-3. They scored four goals, two goals, one goal, five goals, one goal, and no goals. They were 0-17 on the power play, and they gave up two, count them two, 3 goal leads.

Needless to say, this was just a hard set of games to watch. The season was hard to watch as a fan. The Firebirds would find ways to lose games. In previous seasons these were the games that they would find some way to win!

There was an article in the Desert Sun that spoke about how proud Coach Laxdal was of the players and how much effort that they gave. And I agree, they did give a lot of effort and he spoke about how young they are.

And again, they are young, and missed their captain Max McCormick for basically two thirds of the season. But they did have some veteran players out there Mitchell Stephens, Brandon Biro, Cale Fleury and Gustav Olofsson. Unfortunately it was just too much to try and overcome.

One of the things that Coach Laxdal also commented on was exactly how much younger next year's team might be. And so while I am again very excited about watching hockey in six months, which is just so long away. I am lowering my expectations for the 25-26 season even lower than they were this year. I'm really hoping we make the playoffs, but won't be surprised if we don't.

And that's going to be okay ... because even bad hockey is still hockey. And I love hockey, and even when they lose, I love watching the Firebirds.

During the regular season, there was exactly one shutout by Victor Ostman ↩︎

Uptime Kuma and Claude

2025-05-08T00:00:00-07:00

While talking to a colleague of mine he had mentioned something called Uptime Kuma, which is an open source alternative to Uptime Robot. This was interesting for me because I have Uptime Robot checking a couple of my websites and I thought, "Hey, maybe I can use Uptime Kuma as a replacement for Uptime Robot."

I dove into the docs a bit and discovered that there was a Docker image for it, which was nice.

I then used Claude to take the docker run command

docker run -d \
    --restart=always \
    -p 3001:3001 \
    -v uptime-kuma:/app/data \
    --name uptime-kuma louislam/uptime-kuma:1

and turn it into a compose file, which gave me this

version: '3'

services:
  uptime-kuma:
    image: louislam/uptime-kuma:1
    container_name: uptime-kuma
    restart: always
    ports:
      - "3001:3001"
    volumes:
      - uptime-kuma:/app/data

volumes:
  uptime-kuma:

Next, I took the output of that compose file and asked Claude to also generate a caddy configuration for me and it gave me this

monitor.yourdomain.com {
    # Enable compression
    encode gzip zstd

    # Add security headers
    header {
        # Enable HSTS
        Strict-Transport-Security "max-age=31536000; includeSubDomains; preload"
        # Prevent content-type sniffing
        X-Content-Type-Options "nosniff"
        # Referrer policy
        Referrer-Policy "strict-origin-when-cross-origin"
    }

    # Reverse proxy to Uptime Kuma
    reverse_proxy localhost:3001
}

I then SSH'd into one of my servers using Shellfish ¹.

I updated the docker-compose.yml file and my Caddyfile to include what Claude had output.

I restarted my docker containers and didn't get my new container running.

So I took the whole Docker Compose file from my server and I put that into Claude and said,

Hey, is there anything wrong with my Docker Compose file?

It indicated that there were some issues and provided updates for. I made those changes and did the same thing with the Caddyfile. Again, Claude offered up some changes. I applied the recommended changes for the docker-compose.yml file and the Caddyfile stopped and started my docker containers.

I suddenly had an instance of Uptime Kuma. All in all, it took about a half hour from start to finish while I was watching a hockey game ... from my iPad.

I didn't really have to do anything other than a couple of tweaks here and there on the Docker Compose file and a couple of tweaks here and there on the Caddyfile. and I suddenly have this tool that allows me to monitor the uptime of various websites that I'm interested in.

As I wrapped up it hit me ... holy crap, this is an amazing time to live². You have an idea, Claude (or whatever AI tool you want to use) outputs a thing, and then you're up and running. This really reduces that barrier to entry to just try new things.

Is the Docker Compose file the most performant? I don't know. Is the Caddyfile the most secured lockdown thing? I don't know.

But for these small projects that are just me, I don't know how much it really matters.

this is an amazing app on the iPad, highly recommend ↩︎
Yes, there's also some truly horrific shit going on too ↩︎

The Invisible Decision-Makers: Why Systems Ignore Their Users

2025-03-31T00:00:00-07:00

The Origin of Systems

When thinking about systems it's easy to think that they have always been there, or been that way. This isn't true of course. The systems that are in place were put there, by people. People that made decisions. Decisions are what I want to focus on here.

In general when making a decision about the implementation of a system you would want to engage with the stakeholders of that system. This of course implies that you can identify at least some of those stakeholders.

But sometimes there aren't any key stakeholders other than regulations, or best practice, or some other nebulous thing that needs to be met. These are the decisions I really want to focus on.

The Illusion of Success

Take a security system for instance. The basic tenets of the security system are that it keeps 'something' safe. If the thing to be kept safe is still safe after the implementation of the security system then the people that implemented the system can claim success. They can look at the evidence that since the security system was put into place the thing has been kept safe.

Of course, it's entirely possible that the thing was never in danger, and that the previous system was doing just fine. In fact, it could be that the security system is actually making it harder to keep the thing safe. It's just harder to see because all you're looking at are potentially meaningless metrics like. Questions like is the thing safe after implementation of the security system don't take into account if the thing was 'unsafe' before? This can lead you to think that the new security system must be responsible for the safety of the thing.

Something else that can be happening is that the security system has caused the people responsible for keeping the thing safe to work more hours, hire more people,who are oftentimes keeping the security system running.

Questioning Purpose

The more we look into a system like this, the more we might ask, "Why is it there?"

There can be a couple of reasons, but I'll focus on one in particular. The person ultimately responsible for keeping the thing safe can show with some kind of metrics that the thing is safe with the new security system, whereas they couldn't under the previous system. There weren't any reports or metrics that showed what was going on, which is why the system was implemented in the first place.

OK ... so that's how some systems can be put in place.

User-Hostile Systems

What about systems that are hard to use, or maybe actively hostile to their users? How do those get put into place? I would argue that the reason we see many user hostile systems in place is because they are decided upon not by their users, but by their ability to meet regulations, AND their ability to maintain by a support system. The consideration of the user is secondary, or maybe not even thought about.

Think about any Enterprise software you've ever encountered. Would you say that it was a joy to use? Would you say that onboarding was simple, and that new employees loved to use it? My guess would be no.

Why Bad Systems Persist

So if the users don't like it, why is it in place? Two reasons:

It meets some kind of regulation (this could be a government regulation, but it could also be a regulation of a guild, or union, or something else)
It's easy to maintain by the support system

For any software that meets these criteria you are likely to have users that don't like the software, because they are always an afterthought. The primary responsibility of the software developers of these types of systems is always the regulators, and the support infrastructure.

The first because they have to keep producing software that is compliant in order to be sold with a specific rating or seal of approval.

The second because if the support team can't easily support it, they're going to find an alternative solution that they can support.

Conclusion

It's a simple decision of maximizing for the people that enforce the rules (regulators) and that make the decisions (support). The users of the software don't matter. At all.

This is why you will see software for widget processing that could benefit from bulk operations, keyboard shortcuts, or being browser agnostic and they just aren't. The only considerations are: Does it meet the regulations? Is it easy to support? If the answers are yes then the users tend to lose out. They don't matter. If the answer is no, then find a competitor that does and move over to them, even if the current system is loved by your users.

Process, People, and Priorities

2025-03-09T00:00:00-08:00

In every organization, three critical elements determine success: People, Processes, and Priorities. While all are essential, their ranking matters profoundly. Based on my experience across several organizations, I've found that Processes must come first, followed by People, with Priorities anchored firmly at the foundation.

This deliberate ordering—Processes at the top, People in the middle, and Priorities as bedrock—creates the most stable and effective organizational structure. When Processes guide how People work and how Priorities are determined, organizations can avoid the chaos of constant priority shifts, reduce dependency on specific individuals, and create consistent frameworks for decision-making.

Defining Terms

Let's define what each of these mean from an organizational perspective:

Processes - How to solve the problems
People - Who will solve the problems
Priorities - The order in which to solve the problems

Process

In my experience ranking Priorities first leads to lots of changes to Priorities. This week it's shipping a new feature to make all of the buttons cornflower blue ... next week it's adding AI to the application. The week after that it's to mine bitcoin. Priorities shift, and that's OK, but priority driven organizations seem to not have a true defining north star to help guide them, which in my experience that leads to chaos.

Ranking People first sounds like a good idea. I mean, who doesn't want to put People first? I have found however that when People are prioritized first bad things can happen. Cliques can form. Only Sally can do thing X and they're out for the next three weeks and no, there isn't any documentation on how to do that. Management can be lax because that's just Bob being Bob and can lead to toxic work environments.

I think that putting Process first helps to mitigate, though not outright eliminate, these concerns.

Processes help to determine how we do thing X. If Sally is out, that's OK because we have a Process and documentation to help us through it. Will we get it done as quickly as Sally would have gotten it done? No, but we will get it done before they come back.

Processes also help implement things like Codes of Conduct. Again, that won't prevent cliques from forming, and no it won't keep Bob from being a jerk, but it creates a framework to help deal with Bob being a jerk and potentially removing them from the situation entirely.

Processes can also help with prioritization. Having a Process that helps to guide HOW you prioritize can be very helpful. This doesn't prevent you from switching up your Priorities, but it does help to keep you focused on something long enough to complete it. And when you need to change a priority it's a lot easier (and healthier) to be able to point to the Process that drove the deicsion to change versus a statement like, "I don't know, the CEO saw something on Bloomberg and now we're doing this."

Setting up Processes is hard. And in a small environments it can seem like it's not worth it. For example, asking "Why do we have a 17 page document that talks about how Priorities are chosen if it's just a handful of People?" Yes, that IS hard. And it might not seem like it's worth it. But you don't need a big long document to determine a Process on how to change Priorities. It can be as simple as

We are small and acknowledge that change is required. We will only change when a consensus of 60% of the team agree with the change OR if the CEO and CFO agree on the change.

More complicated Processes can come later. But at least now when a change is needed you know HOW you're going to talk about that change!

People

What comes second? I find that People should be next. It's the People that are going to help make everything happen. It's the People that are going to help get you over the finish line of the projects that are driven by your Processes. It's People that will work the Processes.

Once you have good Processes and good People, then you can really start to set Priorities that EVERYONE will understand.

An Example

My least favorite answer to the question, "Why do we do it this way?" is "I don't know."

In my opinion this points to a broken culture. It could be that when you started you did ask questions, but you were shot down so many time for asking that you just stopped asking. It could be that you're not very curious and someone just told you and didn't provide a reason and you just accepted it as gospel that this is the way that it needs to be done.

The reason why this is a toxic trait is that you can have a situation like this occur

While working on a report a requester indicated that the margins weren't quite right and it was VERY important that they be 'just so'. I met with the requester and asked them about the Process and it went something like this:

When I drew out the flow and asked the requester why, they said, "I don't know, that's just how Tim trained me"

I was fortunate that Tim was still at the company, so I called him and asked about the Process.

He laughed and said something to the effect of, "They're still doing that? I only had that in place because of an issue with a fax machine 8 years ago but IT fixed it. Why are they still doing it that way?"

"Because that's how they were trained"

🤦🏻‍♂️

Always understand why you're doing a thing. Always. This points to the need for Process, and why I place it first. Process matters and it helps to inform the People what they need to do.

Priorities

Why are Priorities last? How can something as important as Priorities be last?

I would argue that Priorities should be the bedrock of you organization and they should be HARD to change. Constantly shifting Priorities leads to dissatisfaction, and burnout. It can also lead People to wonder if what they do actually matters. If it's always changing, why should I care about what I'm working on right now if it's just going to be different later today, tomorrow, or next week.

The interplay between Processes, People, and Priorities forms the backbone of any effective organization. By putting Processes first, we create the infrastructure that enables People to thrive and Priorities to remain stable. Good Processes provide clarity, continuity, and a framework for decision-making that transcends individual preferences or momentary urgencies.

When organizations understand that Priorities should be difficult to change—and that a clear Process should govern how and when they change—they protect their teams from the whiplash of constant redirection. This stability doesn't mean rigidity; rather, it ensures that when change does occur, it happens deliberately, transparently, and with organizational buy-in.

Whether you're leading a startup of five People or managing departments within a large corporation, begin by examining your Processes. Are they documented? Do People understand not just what to do, but why? Is there a clear Process for establishing and modifying Priorities? If you can answer "yes" to these questions, you've laid the groundwork for an organization where People can contribute meaningfully to Priorities that truly matter.

Remember: Process first, People second, and Priorities as the bedrock. Get this order right, and you'll build an organization that can handle change without losing its way.

Technical Solutions to People Problems

2025-02-06T00:00:00-08:00

"If you think technology will solve your problems, you don't understand technology and you don't understand your problems"

~ attrib. Laurie Anderson

From a Toot by Jake Rayson

In a previous post, I wrote about how to ask why without sounding like a jerk. This is a slightly related concept (at least in my head).

Sometimes, as technical people, we are asked to solve problems. The more we dig into them, the more we discover that the problem that needs to be solved isn't a technical one but a people one. In many cases, it's just getting two groups to actually talk to one another.

This can be hard and awkward, so people may want to avoid it. Creating a report telling someone they're doing something wrong is way easier. No hurt feelings! However, I've found that the approach tends to create more problems than it solves.

The situation

The situation is a real one, and I'm obfuscating details to help 'protect the innocent'.

At the start of each year, large amounts of new data are needed to be added to a system. The additions are, by their nature, very manual¹ and so the team responsible for them spends much of their time trying to get the data added.

Another team is highly dependent on this new data being added in order to process their widgets². The widgets get loaded into the system and checked to see if the data from team A is complete. If it isn't, then the widget gets flagged. This flag directs the members of Team B to reach out to Team A to get clarification on the state of the data needed to process the widget.

Only, that's not how Team A understands it. While they are furiously trying to update data, there is some basic data that covers roughly 80% of the widget processing data needs that are already available. So, the vast majority of the time, there is no need for Team B to reach out to Team A because the information they need is available in the system to process the widget.

This understanding was either lost or never communicated effectively so Team B would just email Team A with questions about the widget data and then get their answer and move on. This is despite the fact that the information is available in the system for the members of Team B to review!

The leader of Team A asked me if I could 'update a report' to 'remove some of these widgets so Team A could better focus on the work of adding the data'.

I thought that seemed reasonable, so I asked Team B a few questions and then made a bit more discoveries and found out the actual problem, which was that the information needed by Team B was in the system. Team A just needed Team B to do a better job of looking for it and asking questions about the things that were needed instead of everything.

The Solution

I proposed that the leaders from Team A and Team B get together to talk about the issue.

At the meeting, the leader of Team B was horrified to hear what was happening. They had no idea that many emails were going to Team A about questions that the members of Team B should be able to answer on their own.

This is all well and good, but why did it take a tech person to spot this and get the team leadership together to figure it out?

I wish I knew the answer. I think part of the insight I had was the current pipeline of work, how long it was going to create a report, and the need to have the problem solved sooner rather than later didn't line up. At all.

I needed to look for a potential non-technical solution. The other thing that I think happened here is that I wasn't weighed down by any history of interactions between the Teams. I was just trying to gather information. In gathering information I was able to see what the real problem was and that the only solution that made sense was for the two teams to just talk to each other.

The Outcome

During the meeting, Team B committed to retraining staff and helping to make sure that they only reached out when there was an actual question about the data for the widget production. Team A was thrilled with this solution, and now they can focus on getting the data into the system more efficiently and with fewer interruptions. A win-win, all because a tech guy got some non-tech people to talk to one another.

yes I would like to automate this, but one step at a time! ↩︎
Not actually widgets ↩︎

How to Watch a Hockey Game - Reading the Standings

2025-02-03T00:00:00-08:00

This is the fourth part of my How to Watch a Hockey Game Series. You can catch up on previous articles here

Game Outcomes

In many North American sports when reading the standings there are typically just Wins (W), and Losses (L).¹

Hockey is a bit different. When you look at the standings for Hockey you'll see 4 headers:

W: Wins
L: Losses
OTL: Overtime Losses
SOL: Shootout Losses

As discussed earlier in this series, if a game is tied at the end of regulation, a five-minute overtime period is played. If either team scores during this Overtime period then the winning team gets a Win, while the losing team gets an Overtime Loss (OTL).

If they're still tied then a Shootout is played. Once a winner is declared in the Shootout they get the Win, while the losing team gets a Shootout Loss.

Because of this, values are assigned to each type of outcome:

Outcome	Points
Win	2
Loss	0
OTL	1
SOL	1

This might best be shown with a concrete example.

A Concrete Example

Let's say that the Coachella Valley Firebirds have played 39 games so far. They have won 21 games and lost 13 games. They've also played in 5 games that went into overtime and lost. Their overtime losses are one (1) in the Overtime period and 4 in Shootouts. Their record would look like this:

Coachella Valley Firebirds: 21-13-1-4

Points Calculation:

Wins: 21 × 2 = 42 points
OTL: 1 × 1 = 1 point
SOL: 4 × 1 = 4 points

Total: 42 + 1 + 4 = 47 points

The Firebirds play in the Pacific Division of the Western Conference, and the standings might look like this:

Team	GP	W	L	OTL	SOL	PTS	PCT
Calgary	41	27	13	1	0	55	0.671
Coachella Valley	39	21	13	1	4	47	0.603
Colorado	36	21	11	2	2	46	0.639
Ontario	37	22	13	1	1	46	0.622
San Jose	36	20	13	1	2	43	0.597
Abbotsford	37	20	15	1	1	42	0.568
Tucson	37	19	16	2	0	40	0.541
Bakersfield	35	16	14	4	1	37	0.529
San Diego	37	11	20	4	2	28	0.378
Henderson	39	12	25	2	0	26	0.333

Legend: - GP: Games Played - W: Wins - L: Losses - OTL: Overtime Losses - SOL: Shootout Losses - PTS: Points - PCT: Points Percentage

Winning Percent

There are 2 things to look at in the standings: (1) Total Points, and (2) Winning Percent.

The Total Points we've already spoken about so let's review winning percent.

The winning percent is calculated as the Total Points the team has divided by the total possible points that they could have gotten. The total possible points are calculated as the Games Played x 2 (that is, what are the total number of points that they would have if they won every game they played).

That is

    Winning Percent = Total Points ÷ (Games Played × 2)

For example in the table above, we see that the PCT column for the Firebirds is 0.603. This is calculated by the Points (47) divided by GP x 2 (39 x 2 = 78), that is 47 / 78 = 0.603.

The winning percent allows ranking intra-season when teams haven't played the same number of games. After all games have been played, the rankings are determined by the total number of points a team has.²

Conclusion

You should now be able to parse the standings in a Hockey League and be able to tell how well (or poorly) your team is doing.

This is the end of my series (for now). If there are any other burning questions you have about hockey, reach out to me on Mastodon.

Football also has Ties (T) but they are exceedingly rare and are only ever displayed when the first Tie of the season occurs ↩︎
Depending on the league there are tiebreakers, but that's outside the scope of this article ↩︎

How to Watch a Hockey Game - What to Watch

2025-01-31T00:00:00-08:00

In a previous post of this series I laid out some basic rules of hockey. In this post I'll hopefully provide some tips on what to watch during your first few hockey games.

What should I 'watch' though?

This is a tough question and depends on if you're watching on TV or in person.

On TV

If you're watching on TV you're limited by whatever the camera and director are showing you. Hopefully they're pretty good at what they do and they'll help to show you what is interesting. You'll also have the benefit of replays. ¹

Watching the action on TV will be your best bet. The commentators will do a reasonable job of explaining the play. For some of the best NHL broadcasts you'll want to watch a Canadian feed. This might not be an option depending on where you live, but in general, watching a Canadian feed of a Canadian team will be really helpful.

If, for whatever reason, you're watching an AHL game² the best broadcasts to watch, in my opinion, are the Lehigh Valley Phantoms called by Bob Rotruck and Cleveland Monsters called by Tony Brown. Each of these is a single broadcaster doing both the color commentary and the play-by-play ... and they honestly get so excited it's hard to NOT get excited with them.

In Person

For your first in person game, just try and follow the puck as best you can. If for whatever reason you can't do that, pick a spot on the ice to concentrate on, preferably near one of the goalies. Which one? The goalie of the team you're not rooting for is a good choice! Then you can just kind of watch the action there.

Keeping in mind the rules start by focusing on just one rule - either icing or offside - for an entire period. Once you feel comfortable recognizing that rule during gameplay, switch your attention to watching for the other rule in the next period. For example, if you spent the first period watching for icing, spend the next period looking for offside plays.

Hopefully after a full game you're able to see them when icing or offside happen. If not, it just means you'll need to come back and try again 😁.

What not to worry about

Hockey is a fast paced game. No, like really fast. Don't worry too much about anything other than watching for the puck, if you can, and trying to pick up icing and offside. You'll see other stoppages in play when a penalty is called. The refs will make hand gestures to indicate the call on the ice and someone will be sent to the box.

Don't worry about whether or not a fight will break out. They don't always, and if they do, each player will be assessed a major penalty and will spend 5+ minutes in the penalty box.

Don't worry too much about learning the positions. The goalie is an obvious one (that's the person with all of the pads, the bigger stick, and the giant, well painted mask in front of the net), but trying to distinguish between a defender and a center ... like just don't worry about it!

Conclusion

Hockey is an amazing sport to watch, whether in person or on TV. It can take a little bit of time to get used to the fast pace, but hopefully this series has given you some tips to enjoy it and understand what's going on.

and refreshments that are much less expensive! ↩︎
home of my beloved Coachella Valley Firebirds ↩︎

How to Watch a Hockey Game - Game Play

2025-01-29T00:00:00-08:00

Game Structure

Hockey has some stuff in common with live theater. No ... really! 😁

They both have dressing rooms and they both have intermission ... but that is probably where the similarities end.

Each hockey game is split into three 20 minute periods. There is an intermission between each period that lasts 18 minutes. During the intermission the players go back to the dressing room to regroup and chat about the previous period a strategize for the upcoming period.

Out in the arena there are chances for you to get overpriced refreshments, stand in long lines to use the facilities, or just stay in your seat and watch the silly intermission games.

Some examples I've seen of silly intermission games are Fuego Pong (like quarters, but with soccer balls and large 5 gallon buckets), ice bowling where a player is put into a giant slingshot on the ice and hudled towards inflatable bowling pins, and the dress up game.

It's also during this time that the ice is resurfaced by a Zamboni to make it nice and clean for the next period.

If at the end of the third period the game is tied then you're in luck because you get free hockey, also known as Overtime. One thing to keep in mind is that the overtime rules during a regular season game are different than a postseason game.

Regular Season Overtime Rules

At the end of the third period there is a 1 minute 'intermission' and then a 5 minute overtime period starts. The overtime period will feature 3 skaters from each team as well as their goalie.

If a penalty occurs in Overtime (or is carried over from the third period) the period starts with four players on the power play team and 3 on the short handed team.¹

Each team tries to score a goal first. If they do, then they win in overtime. If, at the end of 5 minutes of play, the score is still tied then a shootout happens.

In the shootout each team has 3 chances to score a penalty shot. Essentially a skater from each team has the opportunity to try and score a goal with only the goalie trying to prevent it. If at the end of the three rounds we're still tied, we keep sending out skaters to try and get that penalty shot until one team is victorious. The record for most rounds of a shoot out is 20 rounds in the NHL, and 16 rounds in the AHL.

Postseason Overtime Rules

Postseason overtime rules are a bit different. Basically you just keep adding 20 minute periods until someone scores. Once a team scores they have won that game. The longest overtime in NHL Postseason history went into the 6th overtime and was played in 1936 between the Detroit Red Wings and the Montreal Maroons. The longest AHL overtime was between the Charlotte Checkers and the Lehigh Valley Phantoms which went into a 5th overtime period. This game started at 7:03 pm local and didn't finish until almost 3:00 am local the next day!

In general most hockey games don't get past the first OT period. From The 2006 playoffs through to the 2024 playoffs there have only been 52 games that have gone into a second overtime period (out of 1312).

OK, you've got a few basics 'under your belt'. In the next part I'll try and answer the question, 'What should I watch?'.

essentially it would be a short Overtime period and probably pretty boring ↩︎

How to Watch a Hockey Game - Three Rules

2025-01-27T00:00:00-08:00

I've written a few times before about hockey. I love watching my local sports puck team¹ and really wish more people watched it. So, I'm going to write a beginners guide to watching hockey so that you too, dear reader, can become an avid fan.

Hockey is a pretty fast paced game at the professional level. In the 90s Fox Sports had broadcast rights to hockey in the US and to help its viewers they had a glowing halo on the puck called FoxTrax which allowed fans to more easily find it. This practice was discontinued at some point, and I honestly think it was one of the better innovations that Fox Sports did and really wish that it would make a come back.

The Rules

As a beginner hockey observer there's only three rules that you really need to know to be able to follow the game.

Offside²
Icing
Power Play / Penalty Kill

The Set up

The ice rink can be broken into 3 sections from the perspective of 1 team. Let's assume we have two teams, A and B. Let's root for team A.

The Defending zone - This is where team A's Goal is located. It starts right behind team A's goal and goes to the right toward the blue line
Neutral Zone - This is the center of the ice between the two blue lines; it also contains a red line that is called 'Center Ice'
The Attacking Zone - This is where team A are trying to score. It starts at the OTHER blue line and goes back behind Team B's goal

Offside

Offside is defined as ... actually that's not important. What is important to understand is that a player on the offense cannot enter their Attacking zone before the puck does. If they do, then that player is called Offside. When an Offside happens a face off takes place outside of the Attacking zone (i.e. in the Neutral Zone) where each team will try and gain control of the puck.

Icing

Icing, or icing the puck, is when a player in their half of the ice and shots the puck down the ice towards their Attacking zone and it is NOT touched by anyone before it passes the face off circles in the Attacking zone. When an icing occurs the puck is returned to the defending zone for a face off³. When an icing occurs the team that the icing is called on have to keep all of their players on the ice, that is, they can not send in any substitutions.

Power Play / Penalty Kill

The two rules above, when broken, result in a stoppage of play and a new face off for each team to try to gain control of the puck. Other rules, when broken, will result in a penalty⁴ which sees one, or more, players sent to the Penalty Box⁵. Penalties can either be minor, which result in a two minute penalty, or major, which typically result in a 5 minute penalty⁶.

When a team is on the Power Play they will have 1 or more extra skaters than the other team. The other team's 'missing' players will be in the Penalty Box. The Power Play team, with the advantage, will remain on that advantage until either they score OR the penalty expires. If a team scores while on the Power Play, they are said to have scored a Power Play Goal.

The team that has penalized players is said to be on the Penalty Kill. They are trying to 'kill' the advantage that the Power Play brings to the other team. If the team on the Penalty Kill scores a goal, it is called a Short-handed goal ... because they were short a person, i.e. short handed, when the goal was scored. In the National Hockey League (NHL), American Hockey League (AHL), and most other leagues when a short handed goal is scored the Penalty keeps going until time is over OR a goal is scored by the team on the Power Play. The Professional Women's Hockey Leagure (PWHL) has a rule (which I think is genius) which states that IF a team scores a short handed goal, the Power Play is over.⁷

In the next post I'll talk a bit more about game play.

The Coachella Valley Firebirds ↩︎
in hockey it is not pluralized like in American Football ... even though in American Football it's not pluralized either! ↩︎
This does NOT apply when your team is on a Penalty Kill ↩︎
I'll talk more about various penalties in future a post ↩︎
it's a small room where players are sent to think about what they did ↩︎
There are a few caveats here about game misconduct, but they're not important for an introductory primer ↩︎
Now, there are lots of Nuances to the PP/PK write up above, but you don't need to understand them initially to enjoy hockey. ↩︎

Remember the Colosseum!

2025-01-21T00:00:00-08:00

The Roman Colosseum

After the fall of the Western Roman Empire in 497 CE the Colosseum fell into disrepair. Rightfully so! Who can worry about keeping up a giant megalith made by people centuries ago while you're just trying to figure out where your next meal may come from, or the ranging hordes of barbarians showing up and taking the food you did find!

However, during the medieval period, while Rome's population declined dramatically and many ancient structures fell into disrepair or were repurposed, the Colosseum remained a prominent landmark. There are stories that as the centuries progressed, the inhabitants of Rome forgot who built it. While some fantastical legends did develop around it, the basic historical facts of its construction by the Flavian emperors and its original purpose remained part of common knowledge among educated Romans. For the non-educated Roman's there were lots of misconceptions about the colosseum.

The non-education Romans would have created stories¹ about the large building. It was haunted. It was used for pagan rituals and no good Christian would go in. Folklore would rise up around it. As many of us have seen or experienced, in the absence of information, people will make it up.²

The Story of the Legacy System

OK, but why is this important from a technology perspective?

Imagine if you will a large system, built 10 years ago, by a group of developers, that have all left the organization.

No one left knows how it works, or how to make changes to it. Most people don't even really know WHY it's there in the first place.

There isn't any documentation that can be referred to. Either because it wasn't ever created OR it was destroyed by Barbarians, I mean well meaning IT processes that 'clean up' unused files.

So what happens? The people remaining create stories about the system. Stories like the long timer 'Bob' that once caused the entire system to Crash and then an old copy backup copy had to be restored, and months worth of work was lost.

No one ever saw Bob after that. Now we're all afraid to touch any part of it. We mostly leave it alone, and it leaves us alone.

There are stories about another gray beard that actually built the system, but everyone assumes these are just fairy tales.

The stories tell of this Gray Beard busting out the entire system in a weekend, using nothing but a pin to move the electrons into the proper places to get all of the logic to work as expected.

Of course, no one really believes that story, but it encourages people to never want to have to make and changes to it.

The problem here is that it's running on a server with an OS that hasn't been supported for 7 years and there is security mandate to upgrade 'everything' to be on current software

No one wants to be in charge of this project, but someone is going to have to be in charge of it.

What do you do?

The story above isn't real, at least not for me. But it could be.

How many times have you gotten to a system that is old, no one around has any idea how it was built and people mostly just avoid it? Probably more than once.

But how can we avoid this fate? Do we just keep the old timers on until they (or the system) die?

There are options, and they are some of the easiest things to do, but many people don't like to do them.

What is the answer?

Documentation

Documentation. No really, Documentation. Just write it down. For a new project especially. For an old project? Most definitely.

For new projects it's best to just get into the habit of writing good docs³ as you go. If that's doc strings in a method, or a full fledged Knowledge Management System using a documentation framework like diataxis, then so be it.

But write it down. Write down the why's whenever you can. Use something like an Architectural Decision Dsocument to understand WHY you made a technical decision you made. Maybe it's not the best decision, but it's the best decision given a set of constraints.

For existing projects, it can be more challenging. It's possible that NO ONE that created the system is at the organization. It could be that NO ONE that asked for the system to be created is at the organization.

This leads to a bunch of problems to try to solve, but the journey of 1000 miles starts with a single step.

How do you solve it?

Use the helpful Awareness-Understanding Matrix⁴

	Aware	Unaware
Understand	Known Knowns	Unknowns Known
Don't Understand	Known Unknowns	Unknown Unknowns

That is,

Known Knowns: Things we are aware of and understand
Known Unknowns: Things we are aware of but don't understand
Unknown Knowns: Things we are not aware of but do understand or know implicitly
Unknown Unknowns: Things we are neither aware of nor understand

The Known Knowns may be very small, but it won't be empty.

The Unknown Unknowns might (will probably) be the largest.

The lack of knowledge here represents Risk⁵. Risk to your team, or to your organization. This Risk needs to be handled as much as possible.

Looking at a system with the Awareness-Understanding Matrix can help to risk it properly. Once you've properly risked the system, then you can start writing documentation.

The documentation can take the form of Architectural Review of System X (DRAFT)

The system does these things

Thing 1
Thing 2
Many other things that are still unknown

Sometimes just the act of writing these things out will help you in finding out what you know and what you don't know.

If you're using a documentation framework like diataxis for this, you will want to keep your documentation parts separated (How To, Tutorials, Reference, Explanation). You may start righting a Reference article on the system and realize that you also need to have some, yet to be discovered, Explanation. The issue is that the Explanation still needs to be researched and written.

That's OK! One strategy I've encouraged, and use, is if I'm writing a Reference Article and need to link to a yet to be written Explanation article, is that I'll simply create the yet to be written Explanation article and tag it with Explanation and Stub. This frees me to come back to it later and fill in the details.

The other thing that will need to be done is to figure out who uses the system. Sometimes that's super easy, and sometimes, it's not.

Once you're able to determine who uses the system, you can talk with them about the system and then work to fill in the gaps from above.

Occasionally, you find out who everyone thinks is using the system, and discover that actually, it hasn't been used for 5 years because reasons, and they didn't know who to tell.

Now you can just retire the system using a decommissioning process. You have a technology decommissioning process, right? If you don't, it may be time to look into one!

Back to the colosseum

The inhabitants of Rome never got to a spot where none of them knew why it was built, or who built it. Or even why. But what did happen is that the people with the knowledge may have been parts of groups that were marginalized and therefore their knowledge was discounted or ignored. Because the knowledge was a verbal knowledge and not written down. It was, to use a loaded term, tribal knowledge. EVERYONE just knows the obvious thing.

But the thing is ... obvious things are only obvious in the context they were created. It's obvious what Python is. I mean, why would someone use a snake to write code to get a computer to do a thing. EVERYONE knows I'm talking about the programming language Python ... until they don't.

Just write this shit down. Make sure everyone gets into the habit of documenting. Make the documentation public. And if it's not possible to make all of the documentation public, make as much public as possible.

For the parts that aren't public, make sure they are accessible by the people that will need access to it.

Really, documentation is a means to an end. Sometimes you won't need the documentation. You'll know how the thing works, and it has an obvious API or UI and people just "get it". This can lead to people not writing the documentation because we don't need it.

This is kind of like saying, I've used a seatbelt every day for 30 years and I've never needed it. I don't see why I need to wear it any more.

This might be fine until you're in an accident.

Not writing documentation is fine, until it's needed. And that's the worst time to discover that you need it.

Better to have it and not need it, than to need it and not have it.

such as it being a temple to the sun ↩︎
Brené Brown ↩︎
any docs in this case are good docs! ↩︎
This is in no way an endoresement of Donald Rumsfeld. He was a horrible person ↩︎
Jacob Kaplan-Moss has a great series on Risk ↩︎

Looking Back at the Half Marathon Streak

2025-01-17T00:00:00-08:00

Looking Back at the Half Marathon Streak

How It Started

In February 2012, a half marathon was being held in Palm Springs, and one of my walking friends asked if I wanted to do it... about 5 days before it was set to happen. I said I wasn't interested, and she said "Where is your sense of adventure?" ... though it was in slightly more colorful language. Six days later, I was participating in my first half marathon.

Now, I hadn't really run too much up to that point. A 5K here and there, and for the most part, I wasn't running much during those "runs".

At the start of the half marathon, I looked at my friend that had persuaded me to participate and said something like, "We're just going to walk, right?" She said we had to run at least some of it.

Eight miles later, I was still running and enjoying it. She wanted to walk a bit. So we did, and then she encouraged me to start running again, so I did.

The First Taste of Success

I finished that first half marathon in something like 2h30m, but I was suddenly hooked. I thought, "I bet if I actually trained for a half marathon, I could do better." So I set out to train for a half marathon. I started a training schedule using this plan and was able to complete the San Diego Rock 'n Roll half in a little more than 2 hours. I continued to run medium distances (up to 12 miles on weekends) and really enjoyed it.

I got to a point where I was in the best shape of my life. I had a resting heart rate of something like 50 with blood pressure that was pretty low (almost low enough that my doctor was concerned!) but my labs were good, and I felt GREAT!

The Birth of the Streak

I was looking at half marathons in 2014 and saw the Palm Springs one again, but also saw one in a local city called Desert Hot Springs in December 2013. Then there was the Carlsbad half in January 2014. Another one looked interesting in Zion National Park (sort of) in March of 2014. And then I saw that there was the La Jolla half in April.

It also turns out that if you run the Carlsbad, La Jolla, and America's Finest City (AFC) half marathons, you get a nice triple crown medal. Well, the AFC was in August. I was looking at the calendar and thought, "Holy shit, I might be able to schedule half marathons each month for an entire year!"

The Journey Begins

In November of 2013, I signed up for a couple of the half marathons that were further out (essentially the ones for the Triple Crown medal) and the Palm Springs half.

I wanted to do the Zion run, but it's a 10-12 hour drive to Zion from where I live. I casually mentioned this to an acquaintance who said, "I'd run that with you."

The next thing I knew, I had a pretty good string of runs set up: (1) Carlsbad, (2) Palm Springs, (3) Zion, (4) La Jolla, (5) America's Finest City ... but I still hadn't told anyone, other than my wife, about my crazy idea to run a half marathon each month.

Making the Commitment

One day while on a walk with a friend, I mentioned that I was thinking about doing it and he said, "That's an awesome idea... you should totally do it." We spent most of the walk with me talking about the idea and wavering until I finally just said, "I'm going to do it."

And that was it. I had made a commitment, publicly, about this thing I was going to do.

The Year of Running

All that was left was to finalize each of the runs, and then train. And by train, I mean run. A lot. So much running.

I ended up running 13 half marathons in 364 days, bookending the feat with the Half Marathon in Desert Hot Springs in December of 2013 and 2014.

In all, I ran in the following cities (some still have my results online!):

Desert Hot Springs (twice!) in December (2013, 2014)
Carlsbad in January
Palm Springs in February
Zion in March
La Jolla in April
Menifee in May
Rock n Roll in San Diego in June
Oceanside in July
San Diego (AFC) in August
Ventura in September
San Luis Obispo (SLO) in October
Santa Barbara in November

During that year I ran roughly 1000 miles training, and 170.3 miles for the actual runs.

The Reality of the Challenge

I have never been so tired in all of my life as I was that year, but I look back on it and it was really fun.

At the beginning of the year, I had an idea that I would get better and faster at running. That didn't happen¹ for a variety of reasons, but mostly because it wasn't fun. What was fun was going to different places with my family where the races were being held and hanging out with them. I still have fond memories of being in Ventura and going to a thrift store with my wife and daughter and finding a pair of roller skates that fit my daughter perfectly.

The Memorable Moments

I remember the kitschy B&B in Santa Barbara where my wife and I stayed and walking around Santa Barbara and finding this cool co-op style building with lots of vendors for getting food, drinks, or artisanal handcrafted stuff.

I also remember running the La Jolla half with a low-grade fever and realizing exactly how bad of an idea that was. Also, that hill at mile five on that course was absolutely brutal!

The Why of It All

In looking back on this, I don't really know why I did it other than the hubris of thinking I could, and then telling someone I was going to.

The hardest stretch was the runs in the second quarter. The first one (La Jolla) was the last weekend in April, the Menifee one was in the middle of May, and the Rock n Roll in San Diego was the first week of June. I ran three half marathons in five weeks. I still look back on that and think it was the point that broke me.

The Challenges

I really did want to give up more times than I can remember, but because I had said I was going to do a thing, I was going to do it no matter what.

There was the nerve-wracking part where I was also going to do a run-a-mile every day between Thanksgiving and New Year's, and then on the first run, I twisted my ankle so my adventure almost didn't even start at all!

The Lessons Learned

I look back on this experience and remember that yes, I can do hard things. Yes, they are worth it, but not always for the reasons that you think they will be.

Telling someone you're going to do something hard can help push you forward and keep you accountable, even just to yourself.

Looking Forward

Would I ever do something like this again? Maybe if I was 10 years younger! In all seriousness, I think I would IF I went into it with a bit of a different headspace. When I started this challenge in 2013, it was with the piss and vinegar of a young person who was sure they could conquer the world.

If I did it this time, I would be more mindful. I would try to enjoy the practice of running. I would enjoy the destinations I was going to get to run in. I would share these joys more with my family.

The Support System

Doing things like this are never done alone. I think that was the thing I learned most about this. During that year, there were some sacrifices made that I didn't realize at all. My wife and daughter did a really good job of supporting me and my decision, but it did put some strain on family life.

Work-Life Balance

Something else it taught me, ironically, was a bit of work-life balance. I was the manager for a technical team at the time. We had an upgrade that was set to happen the weekend before my Menifee run in May. Due to an issue with a third-party vendor for that application, we had to push back the upgrade... I had to decide to push back the upgrade which meant it would happen the weekend of my run.

As I was working towards making the decision, my boss and my team all knew what I was trying to do, and they supported not only the decision to push back the upgrade but also encouraged me to go do the run. They told me that they could handle the upgrade without me, and they did.

The Hard Lessons

That Menifee run was also where I discovered, for the first time, exactly how bad dehydration can make you feel. Not an experience I recommend, and not one I ever hope to repeat. There are still parts of Old Town Temecula that make me feel a bit queasy when I visit them!

The Final Reflection

Overall, this was a great experience, and I learned a lot about myself. But I also learned a lot about my friends, family, and coworkers as well.

Would I do it again ... maybe. Would I like to run at least one of those half marathons again? Absolutely. I just need to get back into running shape!

just take a look at my available results above via the links! ↩︎

Year in Review 2024

2025-01-02T00:00:00-08:00

I did my first Year in Review last year and have decided to carry on the tradition to make sure I know what I did!

I've written about themes before, so I won't go over it again here. Below is a high level of what my 2024 themes were

Overall my themes were pretty successful. I do wish I had been a bit more mindful, and am now left to wonder if I shouldn't have gone with Autumn of Mindfulness instead of Fall of Mindfulness, because I seem to have taken a step back on some of my hopes for mindfulness 😄

Professional

Last year I said

In the moment it can feel like I don't really get anything done at work

I've felt this way pretty much every year for probably since I first became a manager, but I'm starting to embrace it a bit more as I get older I guess.

For some context, in 2024 I worked 2235 hours with the following breakdown:

Category	Hours	Percentage
Administration	1193.5	53.4
Meetings	838.1	37.5
Coding	93.5	4.2
Conference	55.5	2.39
Commuting	54	2.42

There were a couple of highlights from work this year though that I wanted to call out

I celebrated 16 years with my current employer which means that my career is now old enough to drive in the US!

My team finally was able to migrate our SQL Database version control from Subversion to Git. I wrote about the migration to git last year in my year in review and this was the last project that needed to be migrated over.

There are still some things to do to help with the migration to make it easier for the teams that work with this project, but the first couple of steps have been completed which is nice.

One of the big things I wanted to focus on was the validation issues that my company had with transmissions of claims data.

Working with a couple of smart and dedicated people we were able to do some pretty amazing things.

One thing to keep in mind with the validation errors is that they have to be corrected, by a person, in order to allow the claims to be transmitted to the Health Plans (which is a major goal of my company)

At the start of the year, the validation error rate for Institutional Claims was 13.7% while the rate for Professional Claims was 8.7%

By the end of the year those rates were down to 1.1% for Institutional Claims and 0.3% for Professional Claims. This represents decreases for 89% and 96.4% respectively.

I'm really proud of what the team was able to accomplish.

Personal

Health

I usually like to run or walk to keep my cardio health up, but I seemed to keep running into one injury or another with my knees, feet, ankles ... whatever. In July I decided to give swimming a try.

Since July 15 I've swum 83,650 yards / 76489.56 meters ... which is 47.5 miles. This absolutely blows my mind because my first swim was only 200 yards, lasted about 10 minutes and I thought I was going to die.

I'm now consistently swimming 3 days a week for about 55 minutes and 2000 yards / 1828.8 meters.

At about the same time I really doubled down on starting a meditation practice. I tend to do about 20 minutes of meditation each day. In 2024 I had 54 hours of meditation.

I wish I could split up my walking and running statistics, but Apple doesn't think these are different and so they are combined in all of the health apps! My combined Walking+Running stats came in at 1015 miles which looks like a lot, but is down significantly from my high in 2019 of nearly 2000 miles. It's also the lowest annual total by far since 2015 (my first full year of tracking)

I think I know what I'll need to focus on in 2025!

Writing

In June I started a writing group with Mario Munuz and Trey Hunner and that helped to keep me motivated and accountable for writing. I didn't write nearly as much as I hoped, but I was able to get out 18 articles. This is the most since 2021 when I wrote 23, but about 1/3 of my high mark in 2018 when I somehow was able to write 44 articles!

Open Source

This year I expanded the role I had in the Django community and I'm really pleased with that.

I started the year off as a Navigator for the amazing Djangonaut.Space program in Session 1, and was able to fill that same role in Session 2.

I joined the Django Commons admin group with Daniel, Lacey, Storm, and Tim. We've been able to onboard 6 libraries!

I also gave a talk at Django Con US in Durham titled Error Culture. As always, my time at DjangoCon US was a blast and I'm looking forward to seeing everyone in 2025 in Chicago!

I also ran for the Django Steering Council. I wasn't successful in making it into the Steering Council, but the five folks that did are all amazing humans and I'm looking forward to the work that they'll do over the course of the 6.x series.

I've also really enjoyed Jeff Triplett's Office Hours. I don't do nearly enough open source work during those office hours, but it's nice to see people and listen in on, and participate in, some great conversations. I'm looking forward to doing this again in 2025

I've also been trying to attend the DSF Office hours hosted by Jacob Kaplan-Moss and Thibaud Colas. These calls are really interesting and allow a bit of a peek into the DSF Board and what's being worked on. Again, I'm excited about attending these in 2025 as well.

Sports Fandom

I post on social media a lot about Hockey. Specifically the local team near my home, the Coachella Valley Firebirds.

They made it to the Calder Cup Finals again this year. And again they played the Hershey Bears. I wrote about the 2022-23 season, and the Calder Cup finals and the heart break associated with losing in Overtime in Game 7 of a championship. I wish I could say that they were able to redeem themselves, but the outcome was the same ... but losing in 6 games instead of 7. That being said, it wasn't nearly as painful this time around.

In all, I went to nearly 50 Firbirds Hockey games (a few on the road, but most at home) and can't really believe it. Watching Hockey live is a lot of fun!

One of the highlights of the off season was running into a few of the players at a local sandwich shop and getting to chat with the captain Max McCormick. I tried to 'be cool', and I think I might have mostly succeeded, but it was a pretty surreal experience.

I also had the luck to get tickets to a game at Crypto.com arena to see the LA Kings play the Seattle Kraken (which is the NHL affiliate of the Firebirds). It was an awesome game to watch due to many of the players for the Kraken being former Firebirds.

The Kraken ended up losing the game 3-2 but it was still a great time.

Finally, the BIG sports win this year was the Dodgers winning their first Full Season World Series since 1988. Because of life I didn't get to watch as many games of the World Series as I would have liked, but I did get to watch game 5 and that made up the missing game 1 ... I think.

Miscellaneous

Music

I got to see a few Concerts in the Park which is always fun. It's free, and typically a pretty nice evening on some cool grass with a stunning view of the sunset over Mt San Jacinto.

I also got to see Weezer and had a good time hanging out with my daughter.

Finally I rediscovered the amazing music of The Tragically Hip from a toot by Greg Wilson and it's brought me a lot of joy to listen to them again. Phantom Power is my favorite album of theirs with so many good songs. Bobcaygeon is probably my favorite on the album, but it can change depending on my mood.

Empty Nesting

As I wrote about here my daughter graduated from High School and started College in the fall. This has been a big change for my wife, Emily, and I. Our daughter is pretty close by so we can visit easily, but we've tried to give her the space she needs to adjust to college life. It's been pretty successful, but it's still a weird experience to walk past her room and not see her.

Home Garden

I've been posting pictures of my lemon tree on Mastodon over the last year. In November I was finally able to harvest about 30 or 35 lemons. The great thing about a lemon tree is obviously all of the lemons. But the hard thing about all of the lemons is trying to figure out what to do with them.

Emily found a great recipe for Lemon & Chili infused Olive Oil so we used that recipe to make about 12 bottles of our own custom olive oil and about 15 cups of Lemonade. These made some pretty amazing Christmas gifts.

Reading

Looking back at my reading for 2024 and I didn't do nearly as much as I would have liked, or think that I should have.

I was able to make it about 8 Chapters into Software Design by Example. It's a great book, but it's definitely not something you just breeze through.

I was able to finish up Practices of the Python Pro. I found it to be a pretty comprehensive book. I'm not much of a book reviewer so I won't bother writing one here. I got value out of reading it, and I think others will as well.

What I am really missing from my reading list for 2024 is fiction. Like any fiction at all. It doesn't look like I read anything that wasn't technical so I'll be trying to focus on fixing that in 2025

Wrap up

Overall 2024 was a pretty good year for me. There were some things that I wasn't and am still not excited about, but I have decided to try and make things better where I can, stand up for what I believe is right, and just keep on trying to be kind and make the world a better place in the ways that I can.

Weezer Live

2024-11-26T00:00:00-08:00

I started college in 1996. In 1997 one of the most influential albums of my early adulthood was introduced to me ... Weezer's Pinkerton.

I hated it.

I wanted the Blue Album again but different somehow, and Pinkerton was NOT it.

However, a weird thing happened. Once I moved into my Sophomore apartment with a roommate that I can only describe as 'hard to live with' I retreeated into two things:

Final Fantasy VII
Music

Final Fantasy VII is a whole blog post on it's own, so I won't try to cover it here, but music ... and specifically Weezer were instrumental to me surviving my 'hard to live with' roommate.

Listening to too much Weezer

I probably listened to Pinkerton 1000 times (no exaggeration) over my time in college (this was not so hard because it's only a 35 minute album!)

I loved that album, and listening to it STILL brings me back to living in a small, shitty apartment in San Luis Obispo with a 'hard to live with' roommate.

I saw Weezer live in March of 2001 at RIMAC in La Jolla at UCSD¹. I still remember the show. Lots of hipster Weezer fans in their skinny jeans, nerd glasses, and a lot of earnestly trying really hard trying to not try to be 'cool'. I was (still am) a bigger guy that can't fit into Skinny jeans to save my life, so this wasn't really my scene, but I really wanted to see Weezer and I didn't want to care about the hipsters.

I also remember needing to be back in San Luis Obispo the next day for work or school or something. So after the show was done at midnight I got in my car and drove the roughly 4 hours back to San Luis Obsipo so I can do whatever I needed to do the 'next' day. It was a long day, but it was awesome. I got to see Weezer.

And for me, that's kind of where Weezer stopped making music. A few months later Weezer released the Green Album and I tried really hard to like it. I did that with Maladroit as well, but meh. Once Matt Sharp left it wasn't Weezer to me anymore².

Life has a way of happening while you're making other plans

After that life got in the way and my musical tastes changed ... but I still really liked the "real" Weezer.

One of the things I really liked doing with my daughter was driving her to her Dance class. We'd each pick an album we thought (hoped) the other would like and listen to it on the drive out.

Of course, I picked Pinkerton at one point and the Blue Album and she thought they were fine. Ugh, I guess fine is better than I don't like it, but still.

Fast forward to October 10th 2024 and Weezer is playing live, in an arena, not even 15 minutes from my house. I bought tickets for me and my daughter and it was pretty surreal. If you would have told me in 2001 at the Weezer concert that I'd see them again in 2024 with my adult daughter I wouldn't have believed you, but I would have thought it was a pretty cool dream to have 😄

The openers were Dinosaur Jr. and The Flaming Lips. I don't have much to say about Dinosaur Jr, but the Flaming Lips put on a great show. Lots of visual interest and excitement.

Then Weezer came on. And the show was amazing. Again, lots of great visuals and set design. One of my favorite was 'Island in the Sun' which had a Giant star in the background of a tropical looking island with a palm tree on it.

Again, the set design and visual aspects of the show were on point.

The only thing that wasn't great was the sound. If I listed to Pinkerton 1000 times, I listened to the Blue album at least 500 times. And the concert was meant to be a 30th anniversary of the Blue Album where they played it in order. And I have to say, for the first 5 - 10 seconds of each song I wasn't sure what song was being played ... it was a bit disappointing.

That being said, getting to see Weezer, with my daughter, was a pretty epic parenting level unlocked style achievement. I'm glad I got to go with her, even if the sounds was a bit muffled for my tastes.

Looking forward to the next concert!

now called Liontree Arena ↩︎
Yes, I've seen the SNL skit. Yes, I totally identify with Matt Damon's character! ↩︎

uv and pip

2024-11-23T00:00:00-08:00

On Sunday November 3 I posted this to Mastodon:

I've somehow managed to get Python on my macbook to not install packages into the virtual environment I've activated and I'm honestly not sure how to fix this.

Has anyone else ever run into this problem? If so, any pointers on how to fix it?

I got lots of helpful replies and with those replies I was able to determine what the issue was and 'fix' it.

A timeline of events

I was working on updating a library of mine and because it had been a while since it had been worked on, I had to git clone it locally. When I did that I then set out to try uv for the virtual environment management.

This worked well (and was lightning FAST) and I was hacking away at the update I wanted to do.

Then I had a call with my daughter to review her upcoming schedule for the spring semester. When I got back to working on my library I kind of dove right in and started to get an error messages about the library not being installed

zsh: command not found: the-well-maintained-test

So I tried to install it (though I was 100% sure it was already there) and got this message

ERROR: Could not find an activated virtualenv (required).

I deleted the venv directory and started over again (using uv still) and ran into the same issue.

I restarted my Mac (at my day job I use Windows computers and this is just a natural reaction to do when something doesn't work the way I think it should¹)

That didn't fix the issue 😢

I spent the next little while certain that in some way pipx or pyenv had jacked up my system, so I uninstalled them ... now you might ask why I thought this, and dear reader, I have no f$%&ing clue.

With those pesky helpers out of the way, pip still wasn't working the way I expected it to!

I then took to Mastodon and with this one response I saw what I needed

@ryancheley Are you running python -m pip install... Or just pip install...? If that's a venv created by uv, pip isn't installed I think, so 'pip install' might resolve to a pip in a different python installation

I went back to my terminal, and sure enough that was the issue. I haven't used uv enough to get a real sense of it, and when I was done talking with my daughter, my brain switched to Python programming, but it forgot that I had used uv to set everything up.

Lessons learned

This was a good lesson but I'm still unsure about a few things:

How do I develop a cli using uv?
Why did it seem that my cli testing worked fine right up until the call with my daughter, and now it seems that I can't develop cli's with uv?

I did write a TIL for this but I discovered that

uv venv venv

is not a full replacement for

python -m venv venv

Specifically uv does not include pip, which is what contributed to my issues. You can include pip by running this command though

uv venv venv --seed

Needless to say, with the help of some great people on the internet I got my issue resolved, but I did spend a good portion of Monday evening un-f$%&ing my MacBook Pro by reinstalling pyenv, and pipx² ... and cleaning up my system Python for 3.12 and 3.13 ... turns out Homebrew REALLY doesn't want you to do anything with the system Python, even if you accidentally installed a bunch of cruft in there accidentally.

Yes this is dumb, and yes I hate it ↩︎
As of this writing I've uninstalled pipx because uv can replace it too. See Jeff Triplett's post uv does everything ↩︎

Migrating django-tailwind-cli to Django Commons

2024-11-20T00:00:00-08:00

On Tuesday October 29 I worked with Oliver Andrich, Daniel Moran and Storm Heg to migrate Oliver's project django-tailwind-cli from Oliver's GitHub project to Django Commons.

This was the 5th library that has been migrated over, but the first one that I 'lead'. I was a bit nervous. The Django Commons docs are great and super helpful, but the first time you do something, it can be nerve wracking.

One thing that was super helpful was knowing that Daniel and Storm were there to help me out when any issues came up.

The first set up steps are pretty straight forward and we were able to get through them pretty quickly. Then we ran into an issue that none of us had seen previously.

django-tailwind-cli had initially set up GitHub Pages set up for the docs, but migrated to use Read the Docs. However, the GitHub pages were still set in the repo so when we tried to migrate them over we ran into an error. Apparently you can't remove GitHub pages using Terraform (the process that we use to manage the organization).

We spent a few minutes trying to parse the error, make some changes, and try again (and again) and we were able to finally successfully get the migration completed 🎉

Some other things that came up during the migration was a maintainer that was set in the front end, but not in the terraform file. Also, while I was making changes to the Terraform file locally I ran into an issue with an update that had been done in the GitHub UI on my branch which caused a conflict for me locally.

I've had to deal with this kind of thing before, but ... never with an audience! Trying to work through the issue was a bit stressful to say the least 😅

But, with the help of Daniel and Storm I was able to resolve the conflicts and get the code pushed up.

As of this writing we have 6 libraries that are part of the Django Commons organization and am really excited for the next time that I get to lead a migration. Who knows, at some point I might actually be able to do one on my own ... although our hope is that this can be automated much more ... so maybe that's what I can work on next

Working on a project like this has been really great. There are such great opportunities to learn various technologies (terraform, GitHub Actions, git) and getting to work with great collaborators.

What I'm hoping to be able to work on this coming weekend is¹:

Get a better understanding of Terraform and how to use it with GitHub
Use Terraform to do something with GitHub Actions
Try and create a merge conflict and then use the git cli, or Git Tower, or VS Code to resolve the merge conflict

For number 3 in particular I want to have more comfort for fixing those kinds of issues so that if / when they come up again I can resolve them.

Now will I actually be able to 🤷🏻 ↩︎

DjangoCon US 2024

2024-11-17T00:00:00-08:00

DjangoCon US 2024

I was able to attend DCUS 2024 this year in Durham from September 22 - September 27, and just like in 2023, it was an amazing experience.

I gave another talk (hooray!) and got to hang out with some truly amazing people, many of whom I call my friends.

I was fortunate in that my talk was on Monday morning, so as soon as my talk was done, I could focus on the conference and less on being nervous about my talk!

One thing I took advantage of this year, that I didn't in previous years, was the 'Hallway Track'. I really enjoyed that time on Monday afternoon to decompress with some of the other speakers in the lobby.

One of the talks that I was able to watch since the conference was Troubleshooting is a Lifestyle 😎 which had this great note: Asking for help is not a sign of failure - it's a strategy.

I am bummed that I missed a few talks live (Product 101 for Techies and Tech Teams, Passkeys: Your password-free future, and Django: the web framework that changed my life) but I will go back and watch them in the next several days and I'm really looking forward to that.

There is a great playlist of ALL of the talks from this year (and previous years) that I highly recommend you search through and watch!

A few others have written about their experiences (Mario Munoz and Will Vincent) and you should totally read those. Some of the

The Food

DCUS via the culinary experience!

Durham has some of the best food and I would go back again JUST for the food. Some of my highlights were

Cheeni
Thaiangle of Durham
Queeny's
Ponysaurus
Cocoa Cinnamon
Pizza Torro
The conference venue food - fried chicken and peach cobbler were my favorite

The Sprints

During the sprints I was able to work on a few tickets for DjangoPackages¹² and get some clarification on a Django doc³ ticket that's I've been wanting to work on for a while now.

The after party in Palm Springs

I left Durham very early on Saturday morning to head back home to Southern California. Leaving a great conference like DjangoCon US can be hard as Kojo has written about.

One upside for me was knowing that a few people from the conference were road tripping out to California and they were going to stop and visit! The following week I had a great dinner with Thibaud, Sage, and Storm at Tac/Quila

Here's a toot on Mastodon with a picture of the 4 of us after dinner

Looking Forward

I just feel so much more clam after the conference, and am super happy.

I'm looking forward to my involvement in the Django Community until the next DjangoCon I'm able to attend⁴. Some things specifically are:

Working on Django tickets
Admin work with Django Commons with Tim, Lacey, Daniel, and Storm
Working on Django Packages with Jeff and Maksudul
Djangonaut Space (if and when they need a navigator but just hanging out in the discord is pretty awesome too!)

I'm so grateful for the friends and community that Django has given to me. I'm really hoping to be able to pay it forward with my involvement over the next year until I have a chance to see all of these amazing people in person again

settings consolidaion ↩︎
docs update ↩︎
27106 ↩︎
I'm working really hard on DCEU but the timing may not work out ↩︎

Django Commons

2024-10-23T00:00:00-07:00

First, what are "the commons"? The concept of "the commons" refers to resources that are shared and managed collectively by a community, rather than being owned privately or by the state. This idea has been applied to natural resources like air, water, and grazing land, but it has also expanded to include digital and cultural resources, such as open-source software, knowledge databases, and creative works.

As Organization Administrators of Django Commons, we're focusing on sustainability and stewardship as key aspects.

Asking for help is hard, but it can be done more easily in a safe environment. As we saw with the xz utils backdoor attack, maintainer burnout is real. And while there are several arguments about being part of a 'supply chain' if we can, as a community, offer up a place where maintainers can work together for the sustainability and support of their packages, Django community will be better off!

From the README of the membership repo in Django Commons

Django Commons is an organization dedicated to supporting the community's efforts to maintain packages. It seeks to improve the maintenance experience for all contributors; reducing the barrier to entry for new contributors and reducing overhead for existing maintainers.

OK, but what does this new organization get me as a maintainer? The (stretch) goal is that we'll be able to provide support to maintainers. Whether that's helping to identify best practices for packages (like requiring tests), or normalize the idea that maintainers can take a step back from their project and know that there will be others to help keep the project going. Being able to accomplish these two goals would be amazing ... but we want to do more!

In the long term we're hoping that we're able to do something to help provide compensation to maintainers, but as I said, that's a long term goal.

The project was spearheaded by Tim Schilling and he was able to get lots of interest from various folks in the Django Community. But I think one of the great aspects of this community project is the transparency that we're striving for. You can see here an example of a discussion, out in the open, as we try to define what we're doing, together. Also, while Tim spearheaded this effort, we're really all working as equals towards a common goal.

What we're building here is a sustainable infrastructure and community. This community will allow packages to have a good home, to allow people to be as active as they want to be, and also allow people to take a step back when they need to.

Too often in tech, and especially in OSS, maintainers / developers will work and work and work because the work they do is generally interesting, and has interesting problems to try and solve.

But this can have a downside that we've all seen .. burnout.

By providing a platform for maintainers to 'park' their projects, along with the necessary infrastructure to keep them active, the goal is to allow maintainers the opportunity to take a break if, or when, they need to. When they're ready to return, they can do so with renewed interest, with new contributors and maintainers who have helped create a more sustainable environment for the open-source project.

The idea for this project is very similar to, but different from, Jazz Band. Again, from the README

Django Commons and Jazzband have similar goals, to support community-maintained projects. There are two main differences. The first is that Django Commons leans into the GitHub paradigm and centers the organization as a whole within GitHub. This is a risk, given there's some vendor lock-in. However, the repositories are still cloned to several people's machines and the organization controls the keys to PyPI, not GitHub. If something were to occur, it's manageable.

The second is that Django Commons is built from the beginning to have more than one administrator. Jazzband has been working for a while to add additional roadies (administrators), but there hasn't been visible progress. Given the importance of several of these projects it's a major risk to the community at large to have a single point of failure in managing the projects. By being designed from the start to spread the responsibility, it becomes easier to allow people to step back and others to step up, making Django more sustainable and the community stronger.

One of the goals for Django Commons is to be very public about what's going on. We actively encourage use of the Discussions feature in GitHub and have several active conversations happening there now¹ ² ³

So far we've been able to migrate ~3~ 4 libraries⁴ ⁵ ⁶ ⁷into Django Commons. Each one has been a great learning experience, not only for the library maintainers, but also for the Django Commons admins.

We're working to automate as much of the work as possible. Daniel Moran has done an amazing job of writing Terraform scripts to help in the automation process.

While there are still several manual steps, with each new library, we discover new opportunities for automation.

This is an exciting project to be a part of. If you're interested in joining us you have a couple of options

Transfer your project into Django Commons
Join as member and help contribute to one of the projects that's already in Django Commons

I'm looking forward to seeing you be part of this amazing community!

DjangoCon US 2024 Talk

2024-10-17T00:00:00-07:00

At DjangoCon US 2023 I gave a talk, and wrote about my experience preparing for that talk

Well, I spoke again at DjangoCon US this year (2024) and had a similar, but wildly different experience in preparing for my talk.

Last year I lamented that I didn't really track my time (which is weird because I track my time for ALL sorts of things!).

This year, I did track my time and have a much better sense of how much time I prepared for the talk.

Another difference between each year is that in 2023 I gave a 45 minute talk, while this year my talk was 25 minutes.

I've heard that you need about 1 hour of prep time for each 1 minute of talk that you're going to give. That means that, on average, for a 25 minute talk I'd need about 25 hours of prep time.

My time tracking shows that I was a little short of that (19 hours) but my talk ended up being about 20 minutes, so it seems that maybe I was on track for that.

This year, as last year, my general prep technique was to:

Give the presentation AND record it
Watch the recording and make notes about what I needed to change
Make the changes

I would typically do each step on a different day, though towards the end I would do steps 2 and 3 on the same day, and during the last week I would do all of the steps on the same day.

This flow really seems to help me get the most of out practicing my talk and getting a sense of its strengths and weaknesses.

One issue that came up a week before I was to leave for DjangoCon US is that my boss said I couldn't have anything directly related to my employer in the presentation. My initial drafts didn't have specifics, but the examples I used were too close for my comfort on that, so I ended up having to refactor that part of my talk.

Honestly, I think it came out better because of it. During my practice runs I felt like I was kind of dancing around topics, but once I removed them i felt freer to just kind of speak my mind.

Preparing and giving talks like these are truly a ton of work. Yes, you'll (most likely) be given a free ticket to the conference you're speaking at — but unless you're a seasoned public speaker you will have to practice a lot to give a great talk.

One thing I didn't mention in my prep time is that my talk was essentially just a rendition of my series of blog posts I started writing at DjangoCon US 2023 (Error Culture)

So when you add in the time it took for me to brainstorm those articles, write, and edit them, we're probably looking at another 5 - 7 hours of prep.

This puts me closer to the 25 hours of prep time for the 25 minute talk.

I've given 2 talks so far, and after each one I've said, 'Never again!'

It's been a few weeks since I gave my talk, and I have to say, I'm kind of looking forward to trying to give a talk again next year. Now, I just need to figure out what I would talk about that anyone would want to hear. 🤔

Summer of Writing

2024-10-03T00:00:00-07:00

In keeping with my themes for 2024 this summer was to be 'The Summer of Writing'.

This theme didn't have a specific post or word count, but I knew I wanted to write more¹.

I had a few things I needed to do to get this started. One of them included starting a writing cohort. I wasn't sure how I would do that but it turns out in early June Lacey was having similar thoughts. Mario and Trey had some interest as well and so we formed a writing group!

We meet every Wednesday (more or less) for about an hour.

I had really hoped that the forming the cohort would give me the encouragement and accountability I needed ... and it HAS!

But I also quickly realized that all I had calendared (really) was the Wednesday writing session, and so I set out to have a 30 minute daily writing session on my own.

Life has a way of kicking you in the ass though when you least expect it.

Roughly 10 days into my 'Summer of Writing' a work thing came up and kind of consumed all of my thought and energy. I realized quickly that something had to give, and so I looked ahead at my Autumn theme and borrowed from it a bit, while still keeping the spirit of trying to write.

The next theme was going to be 'The Autumn of Mindfulness' which included starting a meditation practice so I dove into that. I also decided that I needed to try to find something to do from a physical activity perspective. I live in the desert of southern california so the summers are brutal ( daily highs that can average 110F+) and being outside isn't something I really like, even in the early morning, before the sunrise, the temps can be mid to high 80s ... sometimes even the low 90s.

I decided that I would pick up swimming and going to the gym to help alleviate some of the stress from work.

That, in addition to the writing, seemed to be a good thing to work on.

During my Summer of Writing I only wrote 5 posts (including this one)

Summer of Writing
SSH Keys
How to ask a question without sounding like a jerk (mentioned in the Django News Newsletter #248)
Reflections on Djangonaut Space Session 2 (lots of love on socials)
Mentors (mentioned in Django News Newsletter #251)

The articles had a total of 5237 words and so, from the perspective of writing, I managed to do some writing, but it wasn't really anymore than what I had done during the Spring of Transition where I wrote 4 articles with 3890

However, the ability to transition from one idea to another is something that CGP Grey talks about in his themes and so I don't feel too badly about it ... especially because the meditation and swimming have really been something I'm very proud of.

My meditation practice includes a 20 minute daily session first thing in the morning to help clear my mind for the day.

My swimming routine consists of swimming 3 days a week. When I started I could barely do a 20 minute swim. Just before DjangoCon US I swam 1800 yards in 54 minutes and for the most part didn't stop for a break. If you would have told me that in early July when I started I would not have believed you.

In a weird way the meditation and swimming kind of helped with the writing because it allowed me to stop thinking, which then allowed room for deeper thinking about my writing.

The stress of work has alleviated a bit and so I'm hoping that after DjangoCon US I'll be able to dedicate about 15 minutes a day to writing to start, and then ramp up to 30 minutes (similar to what I did with the swimming) and continue to swim and meditate.

One thing that I've found very helpful is to just add a little bit of a good habit and remove a bit of a bad habit. Sooner or later the bad habit is gone and replaced with the new good habit.

In the Autumn of mindfulness, which I will still try to do, I'll focus on eating right (I kind of eat like a 7 year old whose parents left the pantry stocked with a ton of junk food and then left for the weekend) so I'm going to work to get that under control.

All in all, it's been a successful summer of writing, even if it wasn't what I initially envisioned. But that's OK, and part of life.

As Mark Twain said, "Life is what happens when you're busy making other plans."²

How much more? I don't really know ... just more ↩︎
No he didn't. It was actually Allen Saunders in Reader's Digest in 1957, I just like to attribute everything to either Mark Twain or Abraham Lincoln ↩︎

Mentors

2024-09-16T00:00:00-07:00

Having just finished up my second round of Djangonaut.Space (which I wrote about here) I wanted to write a bit about mentors ... how to find one, how to work with one, and how to be one.

Finding a Mentor

One of the best ways to find a mentor is through a program like Djangonaut Space. You're put into a cohort of other Django / Python programmers with a Captain and a Navigator. A program like this offers up ready made mentors in the form of the Captains and Navigators. Even your fellow participants can act as mentors.

The thing about a mentor, and finding one, is that what you're looking for isn't ONE mentor ... you're looking for a mentor in a specific aspect of life, whether personal or professional. In Djangonaut.Space you'll get a couple of mentors in Python / Django, but you may also find that you get a mentor who helps with thinking about / dealing with / finding developer jobs.

Working with a Mentor

Working with a mentor isn't just showing up and hoping that all of their knowledge in the specific aspect of life you're looking to be mentored on will suddenly flow from them to you like a fountain. You need to do a bit of homework too!

Mentors can provide lots of guidance, but like any guide, you kind of need to know where you're going ... even if it's just a vague direction. Having a goal of

I want to be a programmer

is a bit vague and difficult to help on. A mentor can provide some guidance for that, like

Try Python

but a goal like,

I want to learn programming to help automate some of these things

will definitely lead to more focused advice. Now the mentor can say,

That's great! Check out this book, these blogs, and follow this YouTuber ... also, here are 10 people you might find interesting on Mastodon (or your preferred Social Media platform of choice)

When working with a mentor they might provide open ended advice or guidance and expect that you'll have done something with it. Going back to the previous example, if a mentor offers the advice of Books, Blogs, YouTubers, etc at your next interaction they might ask, "So did you have a chance to check out any of those things".

If your answer is no, that's not the end of the world, but it might signal to the mentor that you're not ready for the mentor/mentee relationship. If your answer is a bit more defined, like "No, work and family have really been crazy, but I've set aside 2 hours this weekend to really check them out" will help the mentor know that you're going to actively try and work on the suggestions made.

Something to keep in mind is that this is a relationship with the mentor. They will try and provide helpful tips and guidance to you, and in return they expect that you'll be acting on those tips or guidance. If you're not willing or able to do that ... that's OK, but maybe this isn't the best time for your mentor relationship to start

How to be a mentor

Going back to my comment above, being a mentor isn't about being the ONLY mentor for a person, but a mentor for that person for a specific thing (or set of things) to help them grow. And that's really the point of mentoring. You want to help someone with their growth so that they can get better at a thing. This will have the strange effect of making you better at that thing as well.

It's easy enough to wave your hands when you're thinking about why something works the way it does, but if you're mentoring someone and they ask you a question you don't know, you are going to do yourself a great service by helping to explain and get them to understand the concept as well.

For example, something that really breaks my brain is mocking. It's just never really stuck with me and every time I need to mock something I'm basically learning it over again. If I had a mentee and they asked about mocking I'd probably get a deer-in-the-headlights sort of look and then say,

You know, it's something I struggle with, but let me write down some thoughts and my understanding on it and talk about it next time.

And, here's the key, next time you meet with them talk through what you learned (again) and show them how you learned it. What docs did you reference? What applications of mocking did you try? How did you try and figure it out?

I think so much of problem solving is learning how to learn. Honestly, if you can be presented with a problem and are able to come up with a solution without much thought then you may not understand the problem as well as you think. You might just be applying a previous solution to the current problem .. essentially trying to make a square peg try to fit in a round hole.

But with mentoring you can help people learn how you learned and to guide them on their journey to discovering things.

Something super important to remember is that it's their journey, not yours.

One thing I do, probably too much, is tell stories to try and get people to understand and remember how things work. I find that stories really work for my brain and help me to retain details that are important, or help to remind me of the ways in which problems were solved.

Honestly, every time someone comes to me with a new problem that I've never seen before my imposter syndrome kicks in like nobody's business! I beat myself up for how stupid I am that I can't solve this problem that I've never seen before.

But slowly, as I work through the problem, I start to see connections to other problems that I've solved. Not the same problem, but similar problems. This helps to get me to a solution ... but short walks help too ... and a good night's sleep.

And this is a prime opportunity for you to take what you've learned, how you've learned it and help a mentee with finding an approach that helps them in similar situations.

As a mentor, you don't need to be a WORLD EXPERT, you just need to be an expert on that one thing in comparison to the mentee. I once heard that an expert is just the person in the room who knows more about a topic than anyone else in that room. You don't need to be a Django Expert at DjangoCon to be a Django Expert at work when trying to introduce Django to developers that haven't seen it before.

Wrap up

Finding opportunities to be mentored can be hard, but a potential good place to start are programs like Djangonaut.Space and similar programs. Other places can be contributing to OSS projects¹

Being a mentor doesn't mean you need to be a world expert, you just need to help one person find resources to help move them along in their journey. If you can do that, then I'd call you a pretty successful mentor!

There are some caveats here, like an open and welcoming community ↩︎

Reflections on Djangonaut Space Session 2

2024-08-29T00:00:00-07:00

A few weeks ago I completed my second session as a Djangonaut.Space Navigator. The Djangonaut.Space program is an opportunity for people to be introduced to contributing to Django and Django adjacent projects.

In this most recent session I was a Navigator for Team Mars with a fantastic Captain Tobe. Our Djangonauts were Andy, Maryam, and Rosana.

Among the 3 of them they took on 7 tickets, pushed 7 PRs and closed 5 tickets.

As part of the program we would meet weekly to talk about any blockers and try and work through them. These meetings also provided a platform to encourage one another.

One week we spoke about being a professional software developer working with Django which was a great conversation.

I really like this program for what it offers both the Djangonauts, and the mentors. I learned so much as part of this program.

As we were coordinating our first meeting I realized that the rest of my team were in time zones that were 7 - 8 hours ahead of mine! I was a bit worried initially that we'd have a hard time finding a common time to meet, but we settled on Wednesdays at noon and this turned out to be pretty perfect for all of us.

Each of our team meetings was similar to a standup where we'd talk about what work had been done the previous week, and any struggles that we were having. The djangonauts on team Mars were absolute Rock Stars. They picked up some pretty gnarly ¹ issues and worked them to completion each time.

Working on a project like Django can be daunting and scary and time consuming. However, the amount that you can learn from working on a large project and code base like this is immeasurable.

Working to form a consensus on an issue or idea, whether it's code or documentation, can be challenging! But as Maryam said in her blog post about her experience with picking up a documentation ticket

To start safely, I picked a documentation change ticket just to get myself familiar with the process. One of my tickets involved updating some wordings in the documentation to make it easier for people to differentiate when a pull request needed a Trac ticket or not. Initially, I thought this would be a simple wording change. However, I soon realised that making changes to Django documentation itself requires a lot of thought and consideration.

This experience reminded me of my early days as a Django user. I loved Django for its documentation - detailed, thoughtful, well-organised, and easy to follow. Now, working on documentation changes as a contributor has shown me how Django achieves such clarity. Significant thought and effort go into making it clear and readable, minimising confusion and maximising understanding for readers.

If you don't know this going in then you can be disappointed or disillusioned with how long something might take to be accepted, or whatever, but a program like Djangonaut Space does, I think, help to ease newcomers into contributing and setting realistic expectations and, in general, enjoying the process.

One thing I tried to really emphasize with my team was that it won't be easy, and it will take some time, but that the effort will pay off with a ticket that has been closed ... and in the worst case you've helped to move it forward.

Another point I tried to keep front and center was the idea that this is a volunteer role and that if you're not having fun it's OK to take a step back. I think we need to hear that more and more, especially given the stress that many developers can be under for their $dayJobs.

I hope that this advice helped them in navigating the tickets that they worked. I also hope it helped to put into perspective what they were doing from a time commitment perspective.

One thing that I really love about the Django community in general, and the Djangonaut.Space community in particular, is how welcoming they are. The community strives to welcome you to be part of it.

BUT even with the welcoming nature, it can still be very hard to pick that first ticket, submit that first PR, and receive that first bit of feedback.

A program like Djangonaut.Space really helps to get people more comfortable with the process of picking and working on a ticket. It also helps to develop long term contributors to the project ... which is amazing.

I'm looking forward to the next time I'll be able to participate and would encourage anyone to get involved, either as a participant, or as a mentor.

Tickets 13376, 35464 and 12203 ↩︎

How to ask why without sounding like a jerk

2024-08-22T00:00:00-07:00

As technical folks working with non-technical folks sometimes the asks that come through are unclear. In order to get clarity on these we want to ask questions to get clarification on the ask, but it can be challenging to not sound like a jerk when we ask. This can happen even IF we do our best to come across in a positive way.

When trying to ask for more details on a project or request I find it's usually best to get to the source of the issue. I like to ask, "What problem are we trying to solve here?" or something similar.

This helps to put you and the requester on 'the same team' trying to 'solve the problem' and not in a potentially negative 'why are you asking me this stupid question' sort of light.

I can't say that I have 'one weird trick' that will always make this not a problem, but recently at my $dayJob I had an experience that might be helpful in seeing how to navigate this particular process.

The problem

I received an email that went something like this

Please see below. It seems that delivery of paper reports via courrier could be automated by sending them to a portal. What are your thoughts?

My initial thought was, "Yes, if we could automate these reports and send them electronically to a portal that would be more efficient."

However, there are some deeper questions here that need to be asked ... like:

Why are we sending these reports in the first place?

Just asking this question though puts us into a potential state of conflict, i.e. it's similar to sounding like you're asking, "why would you do this stupid thing". In order to avoid this I reframed the question into 3 deeper questions that tried to frame 'the problem' and put me and the requester 'on the same team' to 'solve the problem'

What are the reports?
What are the recipients of the reports supposed to do with them?
Do the recipients of the reports find them helpful, or do they just put them in the shred bin?

My first response to the sender was

Ideally any reports that are being delivered on printed paper by courrier would be better served to be delivered via some electronic means. Can you tell me, what are these reports and who are the intended recpients?

I wanted to explicitly ask who the intended recipients were (I work in Healthcare and these reports are 'for the doctors' but they might actually be getting delivered to an office manager, a front desk person, or anyone other than the doctor).

The sender responded back

They are reports that show a key metric for outstanding work left to do for a specific population of their membership. Each doctor (or their office) are free to do, or not do, anything with the information in these reports.

Next I asked if the recipients had been surveyed on the usefulness of the reports and that's when the sender indicated:

Actually, no. It's something that we need to do so that we can potentially consilidate reports and/or eliminate unhelpful reports.

The Solution

At the end we decided that before anywork was done to 'automate' the delivery of these reports, that we really needed to address the contents of the reports and determine which parts of them were helpful, and what parts weren't. Once we have a single report, or potentially a suite of reports, the automation and delivery work could actually start.

By working through and trying to determine the actual problem that needed to be solved by asking questions to help both me and the requester better understand what the real ask was, we saved a ton of development time and have a better path forward for making the information we have more relevant and actionable by the doctors' offices.

Will this work in every situation? Maybe not, but I believe it's a good starting point when trying to solve 'real world' problems in a work setting.

Tech folks have a (sometimes deserved) bad wrap, but we can shed this negative impression by showing the people that request solutions from us that we're both working towards the same goal of solving the problem.

SSH Keys

2024-07-13T00:00:00-07:00

If you want to access a server in a 'passwordless' way, the best approach I know is to use SSH Keys. This is great, but what does that mean and how do you set it up?

I'm going to attempt to write out the steps for getting this done.

Let's assume we have two servers, web1 and web2. These two servers have 1 non-root user which I'll call user1.

So we have something like this

user1@web1
user1@web2

Suppose we want to allow user1 from web2 to access web1.

At a high level, we need to allow SSH access to web1 for user1 on web2 we need to:

Create user1 on web1
Create user1 on web2
Create SSH keys on web2 for user1
Add the public key for user1 from web2 to onto the authorized_keys for for user1 on web1

OK, let's try this. I am using DigitalOcean and will be taking advantage of their CLI tool doctl

To create a droplet, there are two required arguments.:

image
size

I'm also going to include a few other options

tag
region
ssh-keys¹

Below is the command to use to create a server called web-t-001

doctl compute droplet create web-t-001 \
  --image ubuntu-24-04-x64 \
  --size s-1vcpu-1gb \
  --enable-monitoring \
  --region sfo2 \
  --tag-name test \
  --ssh-keys $(doctl compute ssh-key list --output json | jq -r 'map(.id) | join(",")')

and to create a server called web-t-002

doctl compute droplet create web-t-002 \
  --image ubuntu-24-04-x64 \
  --size s-1vcpu-1gb \
  --enable-monitoring \
  --region sfo2 \
  --tag-name test \
  --ssh-keys $(doctl compute ssh-key list --output json | jq -r 'map(.id) | join(",")')

The values for the ssh-keys above will get all of the ssh-keys I have stored at DigitalOcean and add them.

The output looks something like:

--ssh-keys 1234, 4567, 6789, 1222

Now that we've created two droplets called web-t-001 and web-t-002 we can set up user1 on each of the servers.

I'll SSH as root into each of the servers and create user1 on each (I can do this because of the ssh keys that were added as part of the droplet creation)

adduser --disabled-password --gecos "User 1" user1 --home /home/user1

I then switch to user1

su user1

and run this command

ssh-keygen -q -t rsa -b 2048 -f /home/user1/.ssh/id_rsa -N ''

This will generate two files in ~/.ssh without any prompts:

id_rsa
id_rsa.pub

The id_rsa identifies the computer. This is the Private Key. It should NOT be shared with anyone!

The id_rsa.pub. This is the Public Key. It CAN be shared.

The contents of the id_rsa.pub file will be used for the authorized_keys file on the computer this user will SSH into.

OK, what does this actually look like?

On web-t-002, I get the content of ~/.ssh/id_rsa.pub for user1 and copy it to my clipboard.

Then from web-t-001 as user1 I paste that into the the authorized_keys in ~/.ssh. If the file isn't already there, it needs to be created.

This tells web-t-001 that a computer with the private key that matches the public key is allowed to access as user1

The implementation of this is to be on web-t-002 as user1 and then run the command

ssh user1@web-t-001

The first time this is done, a prompt will come up that looks like this:

The authenticity of host 'xxx.xxx.xxx.xxx (xxx.xxx.xxx.xxx)' can't be established.
ED25519 key fingerprint is SHA256:....
This key is not known by any other names.
Are you sure you want to continue connecting (yes/no/[fingerprint])?

This ensure that you know which computer you're connecting to and you want to continue. This helps to prevent potential man-in-the-middle attacks. When you type yes this creates a file called known_hosts

OK, so where are we at? The table below shows the files, their content, and their servers

Server	id_rsa	id_rsa.pub	authorized_keys	known_hosts
web-t-001	private key	ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFwHs8VKWWSH737fVz4cs+5Eq8OcRJRf2ti0ytaChM1ySh2+olcKokHao3fl5G+ZZv4pQeKfCh8ClFP86g7rZN1evu2EFVlmBo1Ked4IwF4UBY2+rnfZmvxeHd+smtyZgfVZI/6ySfe1D+inAqv7otsMsNRRuE4aG0DNEJ39qwFxukGNcDXk9RNVvmwbCc5zT/HN0yMJ6Y7KtfPZgjl5v854VodZkfxsLpah7Bn64zAQr/xDh2KcWbtDrsvTdjNMPY7oW20VoqDs98mA6xAw9RNMI+xotNmivdWdv3BEYj9JyH61euTBQ27HC4LsOPuCOFKBqOwGXiJhpzvJZbNCcvQEztem3kqQFAPLg+4wBInyxnY2i31QX7+2IJs0a4pYTWRSRcrvwBAvi2GlXGltrZ7V6KOLzwBrXLD7XiO3C5kO5fcpanKlm/RdVAxUTjUq159H+v9om8HAgX/pIpYBpPnRrG7setNQVzDNQsxfR/YC0h+f9LWnnaBV6+51IjbaqAPSSf6KYv0AKO5XNlJsSTXNRBZaRvrfr0qllgXU82f9y8Eb0sgjL71wD9Fv24fV0toFW8PH3yOeePC6d7kNqZkFdSBksChzqagZwPudYnVhMmhMYV7k1v831H8WHdGPVRe9Z3BDnSCzf8o8fRS3mSEAJBiT30bXlGWUNopIpsgw== user1@ahc-web-t-001	ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCbx+wTVEcdy2Uu2iB+u6+R8Q0yH9ws92GM6K/XXmAXoUuXylkdJzw9vUeuaZTmGxwGRdp+lLh+vVDmiuzrUPjbkFA7Y1SxfR5lgJu7PviDDZzsFeUo5fqSp6FOC5x75jOjqy6fc68GzOnoxk4WR6EWKWRd+xqdgCTGWiuhfUEl1lw7YN8MUhd1Hi0Ef55ZpH133jCzffWbkLFFInyIwuzG6jaPsobNPRshvg9kUoFwo5WqCx/s8Zk4iVl86yCwoV+pXjiubLylSKF7hb7uDE4Ll8gADOtuXUqmc470yvzSxxI4yaZOFz4Ajo1qZHgscSOxWgb+ZVIOKhGK5ftHPaZ4CCxXuhW5J8L3Aqs0WQeRu9Goof83V/ruZhzgg1vnhmC2511QSS2dL6U7n2JNLtNnXNjeSQ0BGVlY1FuZRczmAxN9nJETmRCdUfiTwKdPS4LdfAwrnckPHKtk1QoFKietLwfbmipU+pGvt6qKpKeRfZ/XGbG+ZiQ7oPiqcYU/eh54IAUxxo9CvVHtn742A4ABqK5+0MJP5VuY3fcDU8dIvA0r4LpxRpG/KSB4yZMUhjf+KR7QUpN3mJIDOKTDAxpGOqpNoD2gTYGpyT13AdrRROpOjOJZJqDiVi6m6r/U+sIgqymsxDqBur5+n4VxvvXbdNd+6vz7AI12WA8I8+0xZw== user1@ahc-web-t-002	NULL
web-t-002	private key	ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCbx+wTVEcdy2Uu2iB+u6+R8Q0yH9ws92GM6K/XXmAXoUuXylkdJzw9vUeuaZTmGxwGRdp+lLh+vVDmiuzrUPjbkFA7Y1SxfR5lgJu7PviDDZzsFeUo5fqSp6FOC5x75jOjqy6fc68GzOnoxk4WR6EWKWRd+xqdgCTGWiuhfUEl1lw7YN8MUhd1Hi0Ef55ZpH133jCzffWbkLFFInyIwuzG6jaPsobNPRshvg9kUoFwo5WqCx/s8Zk4iVl86yCwoV+pXjiubLylSKF7hb7uDE4Ll8gADOtuXUqmc470yvzSxxI4yaZOFz4Ajo1qZHgscSOxWgb+ZVIOKhGK5ftHPaZ4CCxXuhW5J8L3Aqs0WQeRu9Goof83V/ruZhzgg1vnhmC2511QSS2dL6U7n2JNLtNnXNjeSQ0BGVlY1FuZRczmAxN9nJETmRCdUfiTwKdPS4LdfAwrnckPHKtk1QoFKietLwfbmipU+pGvt6qKpKeRfZ/XGbG+ZiQ7oPiqcYU/eh54IAUxxo9CvVHtn742A4ABqK5+0MJP5VuY3fcDU8dIvA0r4LpxRpG/KSB4yZMUhjf+KR7QUpN3mJIDOKTDAxpGOqpNoD2gTYGpyT13AdrRROpOjOJZJqDiVi6m6r/U+sIgqymsxDqBur5+n4VxvvXbdNd+6vz7AI12WA8I8+0xZw== user1@ahc-web-t-002	NULL	\|1\|V6uYGlSiYXpzFAly9RQHybzl07o=\|VUkDfRcKGyUgLdJn+iw6RJE+r68= ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAILpbPHA1jL0MHzBI8qb2X0mHDx3UlrKCdbz1IspvaJW9 \|1\|dshOpqJI2zQxEpj1pleDmtkijIY=\|ZYV8bCeLNDdyE7STDPaO2TzYUEQ= ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnGgQUmsCG23b6iYxRHq5MU9xd8Q/p8j3EyZn9hvs4IsBoCgeNXjyXK28x7Mt7tmfjrF/4jLcq4o2TTAwF6eVQZ4KXoBa73dYqYDmYTVKTwzZL9CsJTWHTsSnU8V/J3Tml+hIFrjZzWP34+lL9xyOVin5R0PT/OCG49ecb5tt2FxTZeyWI47B/bCDGXV9g1tjZ8+mnbLXpIdQ9+6GllRZrEGvXWm6z/U3YHO84dcG0IZJ7QsEaAiLSBC/t83So4MDQgdttm+aHZXds4jej5E3QwUex8JkVVn0X7Nr4yKMDkSk7ABD6AFhpa4ESXysqI33CUaSBROAuu4lmfOkLmyRZK2vQ6soiOW8iBgCEl/q8MSOEpZeAi3faYbUnOpLzLDBcCoAuSDoexrTixxlhJmRDeS3PlcXmzvkJl7RRKUYZZcPQOd2w9ipCIAD1PevNlnmmZcfkRe0RRvAyF1mqcO/x5Ovtq9QLbycFHYfh/3LcPuDOWBtT+mVd5FeNUMsZ6+8= \|1\|HJd+aDFM66x8jJT1zUZV59ceL10=\|PfHQu/Yg35QPBKk7FvNO/46b76o= ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP+XwUozGye03WJ6zC7yoJQaYF8HiUQKmZwnQO0wSxMm9x9nBdPEx1bmyZHHUbMnwQnoeAMmd6hgK6H8hbxzEas=

OK, so now we have set it up so that user1 on web-t-002 can access web-t-001 as user1. A reasonable question to ask at this point might be, can I go the other way without any extra steps?

Let's try it and see!

From web-t-001 as user1 lets run

ssh user1@web-t-002

And see what happens

We get the same warning message we got before, and if we enter yes we then see

user1@yyy.yyy.yyy.yyy: Permission denied (publickey).

What's going on?

We haven't added the public key for user1 from web-t-002 to the authorized_keys file on web-t-001.

Let's do that now.

Similar to before we'll get the content of the id_rsa.pub from web-t-001 for user1 and copy it to the authorized_keys file on web-t-002 for user1.

When we do this we are now able to connect to web-t-002 from web-t-001 as user1

OK, SSH has 4 main files involved:

id_rsa.pub (public key): The SSH key pair is used to authenticate the identity of a user or process that wants to access a remote system using the SSH protocol. The public key is used by both the user and the remote server to encrypt messages.
id_rsa (private key): The private key is secret, known only to the user, and should be encrypted and stored safely
known_hosts: stores the public keys and fingerprints of the hosts accessed by a user
authorized_keys: file containing all of the authorized public keys that have been generated. This is what tells the server that it’s Ok to use the key to allow a connection

Using with GHA

ssh-action from AppleBoy

A general way to access your server with GHA (say for CICD) is to use the GitHub action from appleboy called ssh-action

There are 3 key components needed to get this to work:

host
username
key

Each of these can/should be put into repository secrets. Setting those up is outside the scope of this article. For details on how to set repository secrets, see this article.

Using the servers from above we could set up the following secrets

SSH_HOST: web-t-001 IP Address
SSH_KEY: the content from /home/user1/.ssh/id_rsa ( from web-t-002 )
SSH_USERNAME: user1

And then set up an GitHub Action like this

name: Test Workflow

on:
  workflow_dispatch:

jobs:
  deploy:
    runs-on: ubuntu-22.04
    steps:
      - name: deploy code
        uses: appleboy/ssh-action@v0.1.10
        with:
          host: ${{ secrets.SSH_HOST }}
          port: 22
          key: ${{ secrets.SSH_KEY }}
          username: ${{ secrets.SSH_USERNAME }}

          script: |
            echo "This is a test" >  ~/test.txt

Using this set up we've made the docker image that runs the GHA to be (basically) known as web-t-001 and it has access as user1 in the same way we did in the terminal.

When this action is run it will ssh into web-t-001 as user1 and create a file called test.txt in the home directory. The content of that file will be "This is a test"

I'm using these keys so that I can gain access to the server as root ↩︎

Spring of Transition

2024-06-20T00:00:00-07:00

I've written before about the Theme's that CGP Grey has discussed and I think they're great! I've just recently completed my 'Spring of Transition'.

So what is the Spring of Transition? For me it meant focusing on that last bit of time that my daughter will be living with me and my wife full time.

She just graduated from High School and is getting ready to go off to college in the Fall. I've taken the last quarter to really try and focus on spending quality time with her, and enjoy the last bits of her living here with me and my wife.

One of the things I am eternally grateful for is that when my daughter was a baby/toddler I started a WordPress blog to keep track of all of the adventures we got into. This was 2008 - 2010 and while Facebook was kind of a thing, Instragram was NOT a thing. I used this blog to post pictures with a fun caption of the context of the picture. I would also write a monthly letter to her and recap what fun adventures we had, or what changes I had noticed.

I did this for a couple of years, but then life got in the way and the changes that she was going through were harder and harder to see, and capture, with a camera. This made it very hard to write about as well.

I ended up taking down my site, but I kept a backup of the WordPress XML just in case¹ I would want to use it again.

At one point I stumbled upon a journalling app called Day One that I used to journal. I used it for a couple of years and then found a feature that allowed me to import my WordPress blog data.

I played around with this a bit and finally made the plunge to import the data. It may be the best decision I've ever made with respect to tech.

Over the last several years I focused on trying to journal every day. One of the grear features of Day One is 'On This Day'. After I journal I'll click on that tab and look back at what I've written "On This Day".

The best entries are from the blog. Small reminders of the toddler my (now) adult daughter was.

This has been especially great over the last 3 months as she has wrapped up High School and prepared for College. It's really allowed me to focus on the great times we had, and work to create some awesome new memories.

We didn't do anything exotic, or visit any far off places during this 'Spring of Transition' ... there's already sooooo much to be done at the end of a High School career!

But, I have tried to focus on the things that we do like to do.

We have watched a bunch of Star Wars:

We've also tried to watch the Battle Star Galactica TV show from the early 2000s. This didn't go well.

Another thing that we've been doing is trying to cook meals together. My daughter is a vegetarian, and my wife and I are not, so this makes dinner (and other meals) challenging.

To over come this she found several vegetarian dishes that she thought I would like and we've made them together for dinner. It's been a real treat to see what she thinks of some of these recipes, which are mostly Thai and Vietnamese meals which is not something she would typically eat (she's really big into the various combinations of cheese and starch, i.e. Grilled Cheese Sandwiches, Quesadillas, and Cheese Pizza 😊

We've also been trying really hard to get take out from a Thai place that has many vegetarian options, but haven't been able to make it work just yet. This looks to be something that we'll hopefully be able to do over the summer before she leaves for school.

I think the hardest part about all of this has been knowing that each of the things that she's done will be the last. She had her last dance recital (I've been watching her dance for 15 years). She had her last Girl Scout meeting. She had her last High School class.

Soon she'll have her last night sleeping here before she goes to school.

Obviously I always knew this day would come, but I didn't really think it would get here so quickly. The days are long, but the years are short. I never really understood what that meant until these last few months.

There is now this full grown adult living in my home ... at least for the next few months. But just yesterday she was a silly toddler walking around the house claiming that the elves must have left her milk in the pantry!

I know that her leaving for college isn't the last time I'll ever see her. I mean, she'll still have a room at our house, so she'll want to come back at some point, right? Right?

And it's not like she's going to school on the other side of the country. It's just a short 2 hour drive away. But still ... it won't be the same.

It's just all so different. My wife and I are planning to be empty nesters. Like, what does that even mean? For the last 19 years our daughter has been a part of, and influenced, the lives that we've lead.

I'm also a little nervous about my 'little girl'² going away into the big bad world. I know I shouldn't be though. She is the most thoughtful, capable, intelligent, caring, hard working person I've ever met in my life.

I know she's going to do great in her next chapter.

I just didn't realize that next chapter would come so soon.

this is one of the ONLY times my digital hoarding has actually paid off ↩︎
Fun fact: she's 5'7 with the strength of someone that's been dancing for 15 years so she's not actually little ↩︎

Using justpath to go on a pyrrhic adventure to clean up my PATH

2024-05-24T00:00:00-07:00

A while ago I heard about a project called justpath from Jeff Tripplet on Mastodon. It seemed like a neat project to try and clean up my path and I figured, what the heck, let me give it a try.

I installed it and when I ran it for the first time, the output looked like this

 1 /Users/ryan/.cargo/bin
 2 /usr/local/opt/ruby/bin (resolves to /usr/local/Cellar/ruby/3.2.2_1/bin)
 3 /Users/ryan/google-cloud-sdk/bin
 4 /Users/ryan/.pyenv/shims
 5 /opt/homebrew/Cellar/pyenv-virtualenv/1.2.3/shims
 6 /opt/homebrew/bin
 7 /opt/homebrew/sbin
 8 /Library/Frameworks/Python.framework/Versions/3.12/bin
 9 /usr/local/bin (duplicates: 3)
10 /usr/local/sbin
11 /Library/Frameworks/Python.framework/Versions/3.11/bin
12 /Library/Frameworks/Python.framework/Versions/3.10/bin
13 /usr/local/bin (duplicates: 3)
14 /System/Cryptexes/App/usr/bin (resolves to /System/Volumes/Preboot/Cryptexes/App/usr/bin)
15 /usr/bin
16 /bin
17 /usr/sbin
18 /sbin
19 /var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin (resolves to /private/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin, directory does not exist)
20 /var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin (resolves to /private/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin, directory does not exist)
21 /var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin (resolves to /private/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin, directory does not exist)
22 /opt/X11/bin
23 /Library/Apple/usr/bin
24 /Library/TeX/texbin (resolves to /usr/local/texlive/2024basic/bin/universal-darwin)
25 /usr/local/share/dotnet
26 ~/.dotnet/tools (directory does not exist)
27 /Library/Frameworks/Mono.framework/Versions/Current/Commands (resolves to /Library/Frameworks/Mono.framework/Versions/6.12.0/Commands)
28 /Applications/Postgres.app/Contents/Versions/latest/bin (resolves to /Applications/Postgres.app/Contents/Versions/14/bin)
29 /Applications/Xamarin Workbooks.app/Contents/SharedSupport/path-bin (directory does not exist)
30 /Users/ryan/.local/bin (duplicates: 2)
31 /usr/local/bin (duplicates: 3)
32 /Users/ryan/.local/bin (duplicates: 2)

That's a lot to look at, but helpfully there are a few flags to get only the 'bad' items

justpath --invalid

justpath --duplicates

Running justpath --invalid got this

18 /var/run/com.apple.security.cryptexd/codex.system/bootstrap/ usr/local/bin (resolves to /private/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin, directory does not exist)
19 /var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin (resolves to /private/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin, directory does not exist)
20 /var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin (resolves to /private/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin, directory does not exist)
26 ~/.dotnet/tools (directory does not exist)
29 /Applications/Xamarin Workbooks.app/Contents/SharedSupport/path-bin (directory does not exist)

and running justpath --duplicates got this

 8 /usr/local/bin (duplicates: 3)
12 /usr/local/bin (duplicates: 3)
30 /Users/ryan/.local/bin (duplicates: 2)
31 /usr/local/bin (duplicates: 3)
32 /Users/ryan/.local/bin (duplicates: 2)

Great! Now I know what is invalid and what is duplicated. Surely justpath will have a command to clean this up, right?

Turns out not so much, and for good reasons:

justpath does not alter the path itself, it provides the corrected version that a user can apply further. A child process the one that Python is running in cannot alter the parent environment PATH. As for reading the PATH - justpath relies on what is available from os.environ['PATH'].

So I posted on Mastodon to see how others may have approached this and Jeff replied back

I ran the tool and copied the output before starting from an empty PATH.

Then I ran justpath and added them back one at a time.

mise helped me cut at least half a dozen weird/duplicate path statements alone. I had quite a bit of tool overlap.

I poked around with trying to find where the invalid values were coming from and found this answer

Cryptexes are used to update parts of macOS quickly, without requiring a full rebuild of the SSV (see this answer for details). I don't know whether removing these paths will break the installation of such cryptexes. But if you want to take the risk, you can remove /etc/paths.d/10-cryptex (or move it to a safe place in case you need it later on). PS: Invalid entries in PATH don't really hurt, they primarily slow down (a very little bit) the lookup of new commands run from the shell the first time.

This supported my assumption that the invalid PATH variables were likely due to macOS upgrades (I guessed that based on the existence of invalid PATH variables on other Macs in my house that didn't have a programmer / developer using them)

So, with that I found 2 files that were in /etc/paths.d and moved them to my desktop. Once that was done I restarted my terminal and got this output

 1 /Users/ryan/.cargo/bin
 2 /usr/local/opt/ruby/bin (resolves to /usr/local/Cellar/ruby/3.2.2_1/bin)
 3 /Users/ryan/google-cloud-sdk/bin
 4 /Users/ryan/.pyenv/shims
 5 /opt/homebrew/Cellar/pyenv-virtualenv/1.2.3/shims
 6 /opt/homebrew/bin
 7 /opt/homebrew/sbin
 8 /Library/Frameworks/Python.framework/Versions/3.12/bin
 9 /usr/local/bin (duplicates: 3)
10 /usr/local/sbin
11 /Library/Frameworks/Python.framework/Versions/3.11/bin
12 /Library/Frameworks/Python.framework/Versions/3.10/bin
13 /usr/local/bin (duplicates: 3)
14 /System/Cryptexes/App/usr/bin (resolves to /System/Volumes/Preboot/Cryptexes/App/usr/bin)
15 /usr/bin
16 /bin
17 /usr/sbin
18 /sbin
19 /opt/X11/bin
20 /Library/Apple/usr/bin
21 /Library/TeX/texbin (resolves to /usr/local/texlive/2024basic/bin/universal-darwin)
22 /usr/local/share/dotnet
23 /Library/Frameworks/Mono.framework/Versions/Current/Commands (resolves to /Library/Frameworks/Mono.framework/Versions/6.12.0/Commands)
24 /Applications/Postgres.app/Contents/Versions/latest/bin (resolves to /Applications/Postgres.app/Contents/Versions/14/bin)
25 /Users/ryan/.local/bin (duplicates: 2)
26 /usr/local/bin (duplicates: 3)
27 /Users/ryan/.local/bin (duplicates: 2)

This just left the duplicates that were being added. There are a few spots where applications (or people!) will add items to PATH and there are MANY opinions on which one is The Right Way TM. I use zshell and looked in each of these files (~/.profile, ~/.zshrc, ~/.zprofile) and found that only my .zshrc file contained the addition to PATH for the duplicates I was seeing

/usr/local/bin
/Users/ryan/.local/bin

With that, I simply commented them out of my .zshrc, restarted my terminal and now I am down to only two duplicates duplicate

 8 /usr/local/bin (duplicates: 2)
12 /usr/local/bin (duplicates: 2)

I looked everywhere trying to find where this duplicate was coming from. I tried a few variations of find

find . -type f -name '.*' 2>/dev/null -exec grep -H '/usr/local/bin' {} \;

But the results were only the commented out lines from .zshrc.

I did find this issue on the Homebrew repo and thought I might be onto something, but it was for a different path so that doesn't seem to be the culprit.

I eventually gave up on trying to find the one true source of the duplicate entry (though I suspect that there is something adding it in the same way that Homebrew is can add a duplicate PATH variable) because I found this command

typeset -U path

I added it to my .zshrc, restarted my terminal and voila, a clean PATH

 1 /Users/ryan/.cargo/bin
 2 /Users/ryan/google-cloud-sdk/bin
 3 /Users/ryan/.pyenv/shims
 4 /opt/homebrew/Cellar/pyenv-virtualenv/1.2.3/shims
 5 /opt/homebrew/bin
 6 /opt/homebrew/sbin
 7 /Library/Frameworks/Python.framework/Versions/3.12/bin
 8 /usr/local/bin
 9 /usr/local/sbin
10 /Library/Frameworks/Python.framework/Versions/3.11/bin
11 /Library/Frameworks/Python.framework/Versions/3.10/bin
12 /System/Cryptexes/App/usr/bin (resolves to /System/Volumes/Preboot/Cryptexes/App/usr/bin)
13 /usr/bin
14 /bin
15 /usr/sbin
16 /sbin
17 /opt/X11/bin
18 /Library/Apple/usr/bin
19 /Library/TeX/texbin (resolves to /usr/local/texlive/2024basic/bin/universal-darwin)
20 /usr/local/share/dotnet
21 /Library/Frameworks/Mono.framework/Versions/Current/Commands (resolves to /Library/Frameworks/Mono.framework/Versions/6.12.0/Commands)
22 /Applications/Postgres.app/Contents/Versions/latest/bin (resolves to /Applications/Postgres.app/Contents/Versions/16/

I still want to try and figure out where that extra /usr/local/bin is coming from, but for now, I have been able to clean up my PATH.

So, the question is, does this really matter? In the answer I found the last statement really caught my attention

PS: Invalid entries in PATH don't really hurt, they primarily slow down (a very little bit) the lookup of new commands run from the shell the first time.

So the answer is, probably not, BUT just knowing that I had duplicates and invalid entries in my PATH was like a splinter in my brain ¹ that needed to be excised.

What are your experiences with your PATH? Have you gone to these lengths to clean it up, figured, "Meh, it's not hurting anything, why bother?"

Yes, that is a Matrix reference ↩︎

Trying out pyenv ... again

2024-03-29T00:00:00-07:00

I think I first tried pyenv probably sometime in late 2022. I saw some recent stuff about it on Mastadon and thought I'd give it another go.

I read through the installation instructions at the ReadMe at the repo and checked to see if it was already installed (spoiler alert it was!)

I noticed that I was not on the current version (2.3.36 at the time of this writing) and decided that I needed to update it.

With the update out of the way I tried to install a version of Python with it, starting at Python 3.10 (because why not?!)

pyenv install 3.10

But when I ran it I got an error like this:

BUILD FAILED (OS X 12.3.1 using python-build 20180424)

Which lead me here. There were some comments people left about deleting directories (which always makes me a bit uneasy ... especially when they're in /Library/)

Reading further down I did come across this comment

I had to uninstall and reinstall Home Brew before it returned to work. It concerned the change from Mac Intel to Mac M1(Silicon). See the article below from Josh Alletto to find out why. https://earthly.dev/blog/homebrew-on-m1/#:~: text=On%20Intel%20Macs %2C%20Homebrew%2C%20and, %2Fusr%2Flocal%2Fbin%20.&text= Homebrew%20chose%20%2Fusr %2Flocal%2F,in%20your %20PATH%20by%20default.

The link in the comment was a bit malformed, but I was able to clean it up and get this link. This is where I re-discovered¹ that the way Homebrew is installed changed with the transition to the Apple Silicon.

Now, I got a new M2 MacBook Pro in March 2023 and since I don't use Homebrew a lot AND I didn't really use pyenv for anything, I hadn't noticed that stuff kind of changed.

Following the steps outlined I was able to redo my Homebrew and now have pyenv working. Now, the only question is will it's use 'stick' with me this time?

earlier in the day I was working through a post by Marijke about Caddy and there was a statement in her write up about how Homebrew on the M1 Macs stored files in a different directory, but when I ran the command to check where Homebrew was pointing I got the Intel location, not the Apple Silicon location ... this really should have been my first clue that some part of my set up was incorrect ↩︎

Winter of Learning

2024-03-19T00:00:00-07:00

Winter of Learning Retrospective

Have you heard the good word about themes? If you haven't, take a look at this great video by CGP Grey on Themes and how they can work. For the last couple of years I've been doing yearly themes ... with limited success. This lack of success was entirely due to me not actually reviewing the status of my themes until the end of the year ... and by then it's too late!

This last December I decided that I'd do the themes, but this time I'd do seasonal set of themes instead of one BIG annual theme.

My current theme ended yesterday (March 18th) and this time I'm going to actually take stock of where I am and how 'well' I did.

Since my theme started on December 21, 2023 which is the Northern Hemisphere Winter Solstice, I decided to have a seasonal theme of 'Winter of Learning' with the following things I wanted to learn more about:

Tailscale
Docker
Postgres
CSS
GitHub Actions

To help me keep track of this I dusted off my TIL github repo and started to write down some TILs. Over the course of the 88 days of my Seasonal theme I added 28 TILs. I also had 16 other, more personal, TILs that didn't make it into the repo for a total of 44 TILs. With 88 days that's a 50% hit rate on writing down stuff that I learned.

This is much better than I thought I had done. I'd been pretty down on myself because I had meant to write a TIL every night, but I didn't. I over estimated the number of times I didn't write a TIL and thought I had done much worse on it than I had.

Now, just because I wrote a TIL doesn't mean that it was one of the topics above that I had indicated I would WANT to write about, but that's OK! The point of a TIL is to document some stuff that you learned and the topics above were only ever meant to be guides, not directives.

I think the one thing I learned that I'm more proud of is spending a pretty good amount of time one weekend trying to learn Docker better. During one of Jeff Triplett's office hours I had joked that Docker scared me. And it was me actually saying it out loud that drove me to sit down and figure some shit out. I even had a public notes issue about it!

Overall this Winter of Learning isn't what I thought it would be, but I'm glad I did it. I am going to work to try and keep on writing TILs and hopefully I'll be able to get in at least 2 per week!

That being said, it's now time to prepare for my next seasonal theme ... the Spring of Transition. My daughter is a Senior in High School and is getting ready to head off to college. Now seems like a good time to start getting ready for my wife and I to be empty nesters and so we'll be spending the next 92 days figuring out how we can do that.

An Argument to Realign the AHL

2024-02-24T00:00:00-08:00

What is the AHL

The AHL, or American Hockey League, is a hockey minor league based in the US and Canada. It's widely considered to be the step right below the NHL which is the top Hockey League in North America.

There are 32 teams in the AHL, and 23 of them make the playoffs. The teams play for the Calder Cup.

What are the Calder Cup Playoffs?

The Calder Cup Playoffs are the name given to the AHL team vying for the Championship Calder Cup. The 2023 Calder Cup Playoffs started on April, 18 2023 with 14 teams playing in 7 different series while the other 9 all had byes, that is, they did NOT play in the first round.

One thing to note is that the first round is essentially a play in¹ round of the playoffs, but it's not an evenly divided play-in.

The current format has round 1 with a best of Three Games, rounds 2 and 3 with a best of Five Games, and both the Conference Finals and Calder Cup Finals with a best of Seven Games.

Why Realign?

The ultimate goal is to enhance competitive balance and foster new rivalries, making the playoff race and outcomes more reflective of team performance throughout the season.

The current structure of the AHL divides the entire league into 2 conferences, and each conference has 2 divisions.

In the Eastern Conference, you have the Atlantic and North divisions. The Atlantic has 8 teams, while the North has 7.

Out in the Western Conference, you have the Pacific and Central division. The Pacific has 10 teams, while the Central has 7 teams.

Now an interesting thing about the playoffs is that the top 7 (of 10) teams from the Pacific make the playoffs, while the top 5 (of 7) teams from the North and Central make it. The Atlantic sends its top 6 (of 8) teams to the playoffs.

Each division will have a certain number of teams with a bye-round, that is they don't have the play in the first round.

In the North and Central three teams get a first-round bye, with only 2 teams playing in round 1. In the Atlantic three teams get a first-round bye with 4 teams playing in the first round.

And in the Pacific division, you have 6 teams playing in the first round with only ONE team getting a first-round bye.

So of the 23 teams that make the playoffs, 14 of them play in the first round, and of that 14, 6 come from the Pacific division.

Seems a bit off to me.

This also had the slightly embarrassing (for the AHL at least) aspect of seeing the second-best team in the entire league in the 2022-23 season (the Coachella Valley Firebirds) needing to win a Play-in round to make it into what might be considered the playoffs.

By the time the Calder Cup Playoffs had concluded last year, the Firebirds lost to the Hershey Bears in 7 games. It was the MOST exciting series that I will ever get to see in person, or on TV.

That being said, the Firebirds played 26 out of a possible 27 games during the playoffs last year. The Bears played in 20 out of a possible 24.

If the Firebirds would have had a first-round bye, like the Bears, they would have most likely still played in 23 out of 24 games (3 fewer games than what they actually played) BUT three games can make a huge difference!

I would like to make the case that realignment of the AHL, to a balanced set of divisions, and conferences is not only feasible, and easy, but in the best interests of the AHL.

Before exploring the realignment scenario, I'll outline the proposed changes. The realignment aims to balance the divisions and conferences, ensuring an equal number of teams in each division and a fairer playoff qualification process.

Additionally, if this realignment had happened last year, I believe that the outcomes could have been different (especially given the Firebird Colored Glasses I might be wearing).

There is a post on the AHL Site about realignment. Quoting the League President Scott Howson:

There’s no easy solution. I’m not saying it’ll never happen, but it’s not in the cards right now.

One item stated for a lack of realignment is, "Realignment would break up the Pacific cohort and likely force teams into the Central, and Howson does not see that as likely, given the additional travel burden that it could put on the division’s members."

I'm going to assume this is NOT the case. What I have below is from a "Does this seem like it might work?" perspective. ²

Realignment

With my rationale for the need for realignment of the way, let's get into the actual implementation of the realignment.

I built a Streamlit app to see what a potential realignment would look like that can be found here.

The idea would be:

Move both Tucson and Colorado to the Central Divsiion to get the Pacific Division down to 8 team
Move Grand Rapids from the Central Division (in Western Conference) to the Atlantic Division (in the Eastern Conference) to get the Central and Atlantic to 8

Impact of Realignment on the 2023 Calder Cup Finals

In the NHL there are 2 conferences with 2 divisions of 8 teams each. The playoffs get a total of 16 teams. The three top teams from each division, and then the top two teams from the Conference.

This means that round 1 has 16 teams playing in 8 different series and NO ONE gets a bye.

Let's imagine the 2023 Calder Cup Playoffs with realignment AND a similar style of playoff admission. One difference between the AHL and NHL I'd keep would be to have 5 games in round 1 and 2, and increase to 7 games in the Championship, versus the NHL which has 7 games in every round.

I list the teams below in their proposed (potentially new) division. The number next to the team is the total points each team had at the end of the 2022-23 regular season.

The Eastern Conference would have had the following seeding:

Atlantic Division

Providence (98)
Hershey (97)
Charlotte (86)

North Division

Toronto (90)
Syracuse (81)
Rochester (81)

Wild Card

Springfield (84)
Hartford (81)

The Western Conference would have had the following seeding:

Central Division

Texas (92)
Colorado (90)
Milwaukee (89)

Pacific Division

Calgary (106)
Coachella Valley (103)
Abbotsford (87)

Wild Card

Manitoba (84)
Iowa (79)

Eastern Conference Playoffs

Round 1 Best of Five games

In this section, we'll explore the first-round matchups, highlighting the top contenders and their paths to victory based on past performances and current strengths.

Starting in the Eastern Conference we would have the following round 1 matchups:

Providence (98) vs Hartford (81)
Toronto (90) vs Springfield (84)
Hershey (97) vs Charlotte (86)
Syracuse (81) vs Rochester (81)

To determine the winners I'm going to look at the playoffs last year and then make an educated best guess as to the winners of each series

Providence vs Hartford

Providence lost to Hartford in the Atlantic division semi-finals last year, 1-3 and I see no reason for that to change ... other than maybe Hartford wins in 3 instead of 4. But I'm going to keep it at Hartford wins 3-1

Toronto vs Springfield

Toronto and Springfield didn't play each other in the playoffs last year or in the regular season. Springfield lost to Hartford 2-0 in round 1 while Toronto had a first-round bye and defeated Utica 3-1 in the North division semi-finals. My guess is that Toronto would have won this series 3-1

Hershey vs Charlotte

Hershey defeated Charlotte in the Atlantic division semi-finals last year 3-1. Granted Charlotte had just played 3 games against Lehigh Valley and won that series 2-1, I still think that Hershey defeats Charlotte, only it takes all 5 games. Hershey wins 3-2

Syracuse vs Rochester

Syracuse lost 3-2 to Rochester. Same result this time around I would think

Round 2 Best of Five games

Based on the seedings for the first round, I believe that Hartford would hold a higher position than Rochester

Hershey (97) vs Rochester (81)
Toronto (90) vs Hartford (81)

Hershey vs Rochester

Hershey defeated Rochester 3-2 last year. I believe the outcome would be the same here (although this was in the Eastern Conference finals)

Toronto vs Hartford

Toronto and Hartford didn't play each other last year, though each team did get swept in the division finals. They played each other in the regular season only 2 times, each with the home team winning in OT. With Toronto getting the home nod, I'll extrapolate to say that Toronto wins in 5 games, 3-2

Round 3 (Eastern Conference Finals) Best of Seven

Toronto vs Hershey

Toronto played Hershey 2 times and lost both times. I think that a series like this would be closer, but Hershey comes out on top 4-2

Eastern Conference Champion Review

In this realigned AHL for the Calder Cup finals, Hershey has played 16 out of 17 games, going 10-6 to reach the Calder Cup Finals. When they actually reached the Calder Cup finals last year, they played only 13 games going 10-3 (playing an extra 3 games)

Providence (98)----|
                   |--Hartford (81)----|
Hartford (81)------|                   |
                                       |--Toronto (90)-----|
Toronto (90)-------|                   |                   |
                   |--Toronto (90)-----|                   |
Springfield (84)---|                                       |
                                                           |--Hershey (87)
Hershey (87)-------|                                       |
                   |--Hershey (87)-----|                   |
Charlotte (86)-----|                   |                   |
                                       |--Hershey (87)-----|
Syracuse (81)------|                   |
                   |--Rochester (81)---|
Rochester (81)-----|

Let's take a look out West next

Western Conference Playoffs

Round 1 Best of Five games

In the Western conference we would have had the following round 1 matchups:

Calgary (106) vs Iowa (79)
Texas (92) vs Manitoba (84)
Coachella Valley³ (103) vs Abbotsford (87)
Colorado (90) vs Milwaukee (89)

As with the Eastern Conference bracket to determine the winners I'm going to look at the playoffs last year and then make an educated best guess as to the winners of each series.

Calgary vs Iowa

Calgary didn't play Iowa at all in either the regular season or the postseason last year. Based on the final records, I think it's safe to say that Calgary wins this series, but I believe it's 3-1

Texas vs Manitoba

Texas played Manitoba 8 times in the regular season last year with Manitoba winning 6 of these games. There were several that went into OT which Manitoba won more often than not. I think this goes to 5 games, but Manitoba wins 3-2.

Coachella Valley vs Abbotsford

Coachella Valley and Abbotsford played 4 times, each winning two games. Given the disparity in total points at the end of the year, I think that Coachella Valley wins in five 3-2.

Colorado vs Milwaukee

Colorado and Milwaukee didn't play each other at all in the regular season. I think that this is an even match-up, but I give the edge to Colorado since they're the home team, winning 3-2.

Round 2 Best of Five games

Calgary (106) vs Manitoba (84)
Coachella Valley (103) vs Colorado (90)

Calgary vs Manitoba

Calgary wins this in 5, 3-2. They didn't play each other at all during the regular season, but I think that provincial pride forces the series to 5 games

Coachella Valley vs Colorado

Coachella Valley wins this in 5, 3-2. This is a repeat of the series last year. I believe that it goes the distance again.

Round 3 (Western Conference Finals) Best of Seven

Calgary vs Coachella Valley

Coachella Valley defeats Calgary 4-3

Coachella Valley and Calgary played each other in the Pacific Division Finals last year. It was a brutal series with Coachella Valley winning in game 5 in Overtime. I think that the same result comes out here going the distance.

Calgary (106)------|
                   |--Calgary (106)------|
Iowa (79)----------|                     |
                                         |--Calgary (106)----|
Texas (92)---------|                     |                   |
                   |--Manitoba (84) -----|                   |
Manitoba (84)------|                                         |
                                                             |--CV (103)
CV (103)-----------|                                         |
                   |--CV (103)-----------|                   |
Abbotsford (87)----|                     |                   |
                                         |--CV (103)---------|
Colorado (90)------|                     |
                   |--Colorado (90)------|
Milwaukee (89)-----|

Western Conference Champion Review

At the Calder Cup finals, Coachella Valley has played 17 out of 17 games, going 10-7. When they actually reached the Calder Cup finals last year, they played 19 games going 12-7. Two fewer games to get here.

Calder Cup Finals

In this matchup, Coachella Valley will have up to 4 home games, while Hershey will have up to 3.

Coachella Valley played in 2 fewer games in this scenario, while Hershey played in 3 more games, for a net difference of 5 games.

I think the first two games go as they did last year. Epic drubbings at Acrisure Arena by Coachella Valley over Hershey. I think that the 3 games⁴ in Hershey go 2-1 in favor of Hershey giving the Firebirds a chance to clinch on home ice in game 6 which they do.

Evaluating the Impact of Realignment: A Reflection on Competitive Balance and Missed Opportunities

Going through this exercise doesn't change the outcome of the 2023 Calder Cup Finals. And I don't want this to seem like a sour grapes sort of thing. The Hershey Bears won the Final game last year within the confines of the structure that was set up by the AHL. In that sense, they won it fair and square.

Also, I'm not sure if this realignment had been in place my predictions would have been correct necessarily.

What I think it does point out is an extreme disadvantage that the Pacific division faces in the playoffs. Last year the Coachella Valley Firebirds were the FIRST team west of Austin to make the Finals. They were also the first team to EVER play a playoff game in EVERY timezone that the AHL operates a team in.

Coda

I was bummed that the Firebirds lost in Game 7 last year. I wrote about it just a few days after it happened.

It was the most exciting sporting event I've ever seen, either in person or on TV. I'm not sure anything will ever be that intense and exciting.

I really wish they would have won, and this shows that they just might have been able to if the conference, divisions, and playoff seedings were a bit more balanced.

preliminary round that occurs before the start of the official playoffs. It is typically used to determine who gets the last spots in the playoff ↩︎
A full analysis is something I'll be looking at for another post in the future. There are lots of other items to look at like (1) Schedules, (2) Travel, and (3) Rivalries ↩︎
Coachella Valley is abbreviated CV in some cases below to save space ↩︎
In all honesty the 3 games in Hershey last year were games where the Firebirds seemed a bit tired. There was also some officiating that seemed a bit dubious (an offside that didn't get called that led to a goal that, IMO, shouldn't have counted) ↩︎

Year in Review 2023

2023-12-31T00:00:00-08:00

I've never done a year in review, but this seems like a good a time as any, right? I had a rough outline, but after reading the great Year in Review from Tim Schilliing, Paolo Melichore, and Velda Kiara, I was inspired to actually finish mine.

Professional

In the moment it can feel like I don't really get anything done at work. Looking at my time tracking stats, I do spend A LOT of my time in meetings (nearly 40%) and administration (almost 45%) which is expected for someone in management I suppose, but I really do miss getting to write code more often.

That being said I was able to complete some pretty significant projects at work with the help of my team that I'm really proud of.

Migrations

Change is hard, and we underwent a few BIG technology changes that have gone really well.

The first big change implemented was to migrate from a few Atlassian products (JIRA and Confluence) to YouTrack. I know there are lots of people out there that HATE JIRA, but I loved it and my team liked it. I think that a big reason for that is when JIRA wasn't doing what we needed it to do, I was able to make changes to it. We didn't have to pass it through some change control committee, or get buy in from some high level manager. We just made it work for us ... and it really did work well for us.

The reason we had to migrate from these products was that Atlassian announced in February of 2021 that they would end-of-life the server versions at the end of February 2024. I looked to see if we could migrate to one of their data center versions, but because I'm in Health Care any solution 'in the cloud' needs to be HIPAA compliant. While Atlassian does offer HIPAA compliant versions, you need to have 500+ users for that solution. My organization has 50.

I spent two years trying to figure out how we could keep JRIA and/or to find something that could replace what we had in JIRA and the best solution I could find was JetBrains' YouTrack.

We've been on YouTrack since the end of May and while there are still some features that I miss (support for Mermaid Diagrams, ability to embed the content of one Confluence Article into another Article, automatic linking between JIRA issues and Confluence Articles) overall the workflow parts of YouTrack for issue tracking are much better than JIRA. Easier to set up, easier to maintain.

Another change that we made was changing our Version Control System from Subversion to git, hosted on Azure DevOps. This involved all three of the teams in my department and proceeded in a staged approach over the course of about 3 months. I also helped another department migrate from Subversion to git.

The biggest challenge was the SSIS packages used in our ETL processes, and the database objects.

The SSIS packages took 3 attempts before it stuck, but the ETL devs were positive with each unsuccessful attempt and we finally got over the hump in early December.

The Database objects are unfortunately still in Subversion. This is a limitation of our current tech stack. Migrating to git requires that each developer have their own version of the database but we don't. Honestly the way we have it set up now is something I'd really like to change, but that's a story for a different time.

In all we migrated 25 repositories from Subversion. There is still more work to do with the Web Developers to update our CICD process to fully leverage Azure DevOps, but small steps can make for big changes over time. No need to rush if we have a working CICD system (even if it's kind of Frankensteined together at this point).¹

With this migration to git we were also able to integrate our issue tracking system (YouTrack) with our VCS. It's nice to see commits automatically 'connected' to the issues in YouTrack.

Another thing that I've been able to work on is getting more and more Python enabled for various projects. We have a Django App that we use to manage 'administrative' tables in our MS SQL database, and we've been able to integrate Python in some of our SSIS packages for ETL.

Speaking

One of the goals that I had from my last annual review was to engage in two public speaking activities. While I give lots of presentations at work, they're all via Zoom so the idea of getting up in a room full of strangers and talking was both exciting and terrifying.

The first conference I spoke at was the KLAS Points of Light conference in May in Salt Lake City (only about a week after PyCon US). The talk was limited to 10 minutes and I had 2 co-speakers so I was limited to about 3 minutes of talking time (and if I said I spoke for 90 seconds that would be pretty generous). That being said, I did get up on stage and spoke to a room full of about 200 strangers (and nearly threw up!)

The absolute highlight of my speaking engagements this year was speaking at Django Con (which I wrote about here and here). I won't write more about it, but I had such a great time giving that talk!

Certifications

I was able to achieve a couple of certifications this year. The first was the Google Cloud Platform Cloud Architect Certificate. I wrote about the experience here.

Another certification I achieved was the Certified EDI Academy Professional. Initially I did this mostly because the cost of the classes to work on the certificate for 2 participants versus 3 participants was only $100 extra and there were 2 people in my department that had asked about working on the certification. Since my department is in charge of EDI 'stuff' and I'm in charge of the department it kind of made sense that I should get it too.

While I didn't think it would be super beneficial and did it mostly just because I have been surprised at how useful it's ended up being. Seeing what's possible with EDI in Healthcare has allowed me to work with the EDI Analysts in my department more effectively AND has helped us all to better identify opportunities for automation

Misc

Above I lamented the lack of time to program above, but one thing I was able to work on was a refactor of an Airflow DAG from 2000+ lines down to 150 lines. This was thanks to the DjangoCon Tutorial Django ❤️ Airflow lead by Sheena O'Connell.

This is also the first year since the start of the pandemic that I've gone into the office on a (mostly) regular basis. While it's mostly like working from my home office (lots of Zoom meetings) it is nice to have a different bit of scenery (the new arena where hockey is played is visible from my desk when I look out the window).

In November I also got my first promotion in 7 years which was nice. I went from being the Regional Director to Senior Regional Director.

Finally something I was really excited and proud about was the rating my management team and I got for Employee Satisfaction. This was the first full year that I had two people other than me in the management team and I think that really helped. The satisfaction rating came back at 95%, the highest my department has ever gotten.

Personal

Health

At the end of last year I completed the Running Challenge which lead to me participating in my first organized run (the Panther 5K) since 2018². I had hoped that this would get me back into running and that by the end of 2023 I would have been able to run a half marathon.

These hopes were dashed in April when I contracted COVID (for the second time since the start of the pandemic) and I wasn't back to feeling like myself until late May. Now, in most places of the country late May might be a swell time to start running, but in the Coachella Valley it's already push triple digit highs so I had a hard time getting motivated to start running again when it was that hot.

I started the Running Challenge again this year, but 24 days into it I got a really bad cold that basically is only now (nearly 2 weeks later) truly disappearing. I haven't run in those 2 weeks, but am looking forward to starting rucking and then running again in 2024.

Django

I mentioned above that I spoke at DjangoCon US this year in Durham, but before the conference started I got to see my youngest step brother and his wife at their (new to me) house³. It was a great way to start an amazing week in Durham which is one of the more walkable cities I've been to.

Another bonus was a chance encounter with Ronard Luna (whom I met at DCUS 2022 in San Diego) and some of his Caktus colleagues after day one of the conference. We went and got (really good) Thai that night, had some great conversations and I got to meet some more amazing Django people.

Towards the end of the conference Jay Miller interviewed me about my talk and that was super awesome. I was nervous at first, but Jay (and Dawn) did a great job of making me feel at ease 😁

I also spent time working with Jeff Triplett and Maksudul Haque on DjangoPackages which has been fun and a great learning experience. I'm looking forward to continuing that work next year!

Finally, towards the end of the year I interviewed and was accepted to be one of the Djangonaut.Space Navigators. I'm really looking forward to working with the Djangonauts on my team, as well as my Captain Nishant Aggarwal.

Reading

I had a goal of increasing the diversity (both in style and authors) that I was going to read this year⁴. To this end my daughter Abby helped me by putting together a list of books by Author's to get me out of my reading rut.

I kind of fell off the reading wagon in the last quarter of the year, but I was able to read some really good books that I wouldn't have found otherwise:

American Gods: Neil Gaiman
Scythe: Neal Shusterman
Renegads: Marissa Meyer
Don't Read the Comments: Eric Smith
An Absolutely Remarkable Thing: Hank Green
The Thousandth Floor: Katherine McGee
Legendborn: Tracy Deonn
Mistborn: Brandon Sanderson
War Girls: Tochi Onyebuchi
The Poppy War: RF Kuang

I also read a few books in the Rise of Mankind Series by John Walker⁵

Raid
Conflict

Writing

I only wrote nine articles this year (including this one). It sure feels like more, but in looking back I didn't write my first post until April, and then not again until July. It was really in the last 3 months (since DjangoCon) that I really started to write more with 2 in October and November and three in December.

I'm looking forward to writing more in 2024 with the goal of one article per month. I've started already with trying to write up one TIL a day. This is part of a large theme⁶

Hockey

On December 18, 2022 AHL Hockey made its way to my home town. The best part is that the arena they play in is only 10 minutes from my house so I went to a lot of hockey games.

So far this season isn't going like I had hoped, but a few highlights from last season were:

Getting to see a triple overtime game against the Calgary Wranglers that ended with the Firebirds winning
A game 7 of the Calder Cup finals going to over time

While the 3OT game ended with the good guys winning, the game 7 OT ended with them losing. It was heart breaking, and I wrote about it here, so I won't go over it again. That being said, even though they lost, the fact that I got to go to a Game 7 for a championship was already mind blowing. The fact that it went into overtime was more so. I did a bit a research and it was the first Game 7 OT championship game in either the AHL or NHL since the early 50s, so it was kind of neat to be a part of history.

I've gotten so into the AHL that I've written a silly scraper that dumps data into a datasette instance on vercel.

At the time of this writing the Firebirds are 9 points behind the pace they had last year.

With that, it's still pretty awesome that I get to watch hockey live a couple of times a week and don't have to travel hours to do it.

House

When my wife Emily and I bought our house in 2009 we were surprised that it was on septic and not connected to the sewer. But then we learned that the unincorporated part of the county we live in that's not unusual. Every few years I call one of the local plumbing companies that is highly regarded to empty my septic tank.

This was the year to have the tank emptied and when they came out to empty it, we discovered that the tank was collapsing on itself and would need to be replaced.

Now, this is not an inexpensive expense⁷ but also not totally unexpected. What was unexpected was to find out that because our house was within 200 feet of the sewer line we were REQUIRED to connect to the sewer.

After contacting 12 approved contractors we were able to get one under contract and they got us connected to the sewer. It cost WAAAAAY more than I think anything should⁸, but it's done now so one less thing to worry about going forward

But the silver lining in that is I finally felt comfortable getting a lemon tree in my front yard and it brings me lots of joy. ⁹

Family

This fall my daughter Abby started her Senior year in High School. This is a mind blowing stage in life. It means that this time next year Emily and I will officially be empty nesters.

In preparation for the transition to College we have done a lot of College tours. These have mostly been short weekend trips, but it's been nice to get out there and visit new / different places.

Before the pandemic my family and I would take a stereotypical American style family road trip. We haven't done it since, but we were hoping to do something big this summer.

Those plans were derailed when the sewer bill came in, but the college tours, and a nice long weekend trip to Julian made up for the lack of a BIG trip.

I mentioned above the Hockey games I've been able to see at Acrisure Arena, but one of the extra benefits of having an arena where they play hockey is that they will also play music. I was only able to go to one concert (Paramore with Abby), but Emily and Abby were able to see several shows including Shania Twain, Lizzo, and Pentatonix.

We also live relatively close to LA so we were able to see a couple of events at the Staples Center (I refuse to call it by it's new name) including SZA (all three of us plus a friend of Abby's) and a Kings game (just Emily and me).

Abby was also able to see the last show of Taylor Swift's Eras tour at SoFi Stadium which was a bit stressful as she did it with a group of friends and an adult cousin of one of those friends (that we didn't know) but she had a great time and had a smile as big as any I've seen on her in a while for a few days after.

Emily and I also went down to the Palm Springs Pride parade and got to see 10,000 Maniacs with their new lead singer (Leigh Nash from Six Pence None the Richer)

We have also really started to take advantage of the space in our back yard as a family. As a 15 year work anniversary gift I received a projector TV that we've set up outside. We also got a fire pit to keep us warm in the frigid Desert Winter Nights (I mean, it gets down to a low of like 50 by the time I go back inside 🥶) and reminds me of this meme.

Tropical Storm Hillary

I grew up in the Coachella Valley, and except for a 10 year period (mostly in my 20s) I've lived here my entire life. I've seen Haboobs, felt Earthquakes, seen smoke from nearby Wild Fires, and a couple of pretty bad rain storms (like the Valentine's Day massacre, and the Storm Cell that wouldn't move) ... but I NEVER thought I'd experience a Tropical Storm (which was very nearly a Hurricane) but this year we did.

It was a stressful day but at the end of it we can out unscathed. We were fortunate that we didn't have any property damage, but others weren't. There are still areas of the Valley that are trying to rebuild after the flooding that the storm brought.

Conclusion

When I started writing this I didn't think i I'd have that much to write, but looking back I see that I did!

I'm glad I did this and hope that future me will find some benefit from it. Hopefully 2024 me won't procrastinate writing this until the very last day ... but he probably will.

That's just the nature of these things, right?

Our current stack involves commits to Azure DevOps which is picked up by TeamCity and then deployed using Octopus Deploy ↩︎
That year I ran the LA Marathon in March, and in July I tore a muscle in my left hamstring ↩︎
They've been in the house for almost 9 years! ↩︎
I read mostly Sci Fi written by people that mostly look like me ↩︎
These aren't particular good or well written, but I was in between books and they were on my kindle so 🤷🏼‍♂️ ↩︎
more on that in the next article 😁 ↩︎
Average costs is about $15,000 ↩︎
Close to $30,000 ↩︎
When Emily and I were looking to buy a house we only had three requirements: (1) It couldn't be behind a gate; (2) it couldn't have a pool; (3) it had to have a citrus tree, preferably lemon. We were able to get 2 of the three when we bought the house and it only took 13 years to get the citrus tree! ↩︎

So you want to give a talk at a conference?

2023-12-15T00:00:00-08:00

Last October I gave my first honest to goodness, on my own, up on the stage by myself talk at a tech conference. It was the most stressful yet fulfilling professional experience I've had.

Fulfilling in that I've wanted to get better at speaking in public and this helped in that goal.

Stressful in that I really wanted to do a good job and wasn't sure that I could, or worse, that anyone would care about what I had to say.

Well, neither of those things turned out to be true. I did get a lot of good feedback which tells me I did a good job, and people were very encouraging for the words that I had to say, so people did care.

My presentation went so well that I was even interviewed by Jay Miller.

You can see my actual talk here, but I thought it would also be interesting for you to see how I got here.

Submitting the idea

I submitted my talk idea for DCUS 2023 in May and it was selected in June. That gave me roughly 3 months to get my loose outline of an idea into a 45 minute talk.

Brain storming how the talk would go

I have tried to get a better workflow for brainstorming ideas in general, but I really wanted to up my game for this talk. To that end I used the Story Teller Tactics cards to help determine the path of the story I would tell in my presentation.

That helped when I got to mind mapping¹ my talk.

The use of the Story Teller Tactics, combined with my mind map, lead to a starting point for creating my presentation

My 'Oh Sh%t moment'

Back in early July I was browsing Mastodon (instead of working on my presentation) and came across a link to an article with the title How To Become A Better Speaker At Conferences. I saved it to my read it later service and went on browsing. A few weeks later I actually read the article (around July 25).

This bit of advice got me a little worried:

On a practical level, a 45-minute talk can take a surprisingly long time to put together. I reckon it takes me at least an hour of preparation for every minute of content.

Yikes! That means I would need to prepare and rehearse and prepare for 45 hours over the course of 14 weeks (almost 4 hours per week on average). So I set up a schedule for how I would meet this requirement

Working on the presentation

I spent all of August and the early part of September working on my presentation for about 3 hours a week. Adding slight tweaks to it here and there. I conducted a dry run with my team at work to assess my presentation's progress.

The dry run went fine, but it was clear that my presentation was missing something.

Asking for feedback

Django Con offers up mentors to help you work on your talk. If you're newer to giving presentations, I highly recommend engaging with one of them. The feedback they provide is priceless!

I had the great fortune to reach out to Katie McLaughlin who had just given a talk at PyCon Australia titled Present Like a Pro. I watched that before reaching out to her and she gave some very good advice on presentation.

Getting serious about preparing for the talk

As I said, I had done a dry run with my team at work and it was fine but I could tell that the way I was working on my presentation wasn't getting it to where I wanted it to be. So I decided to go all in on practicing and trying to make it the best I could. In order to accomplish that I believe I would need to engage in deliberate practice by actually giving my presentation. This was a breakthrough moment for me in improving the presentation and my deliver of it.

In order to have deliberate practice I set up the following routine:

Give my presentation and RECORD it
Watch my presentation and make notes about what needed to be improved
Update my presentation based on the notes from #2
Go back to step 1

Steps 1 - 3 were done on different days. For example, on Monday I would record me giving the presentation; on Tuesday I would watch the presentation and make notes; on Wednesday I would update the presentation based on my notes from Tuesday; on Thursday I would start over.

I did this a total of 7 times over 21 days. Two of these times when giving the presentation I gave it to a 'live' audience and was able to get feedback from them on various parts of the presentation.

I did one final dry run on October 13th (the Friday before my presentation was to happen for real)

That Friday was the last time I even looked at my presentation before giving it. I know some people will talk about making updates on the plane ride to the conference, or the night before, or the hour before, but that would stress the crap out of me, and I was already stressed out enough!

Giving my talk for Real

On Monday October 16th I gave my talk in front of people, in real life, for the first time. Here I am up on stage with the crowd in the background

All in all it was a really fulfilling experience, but it was pretty hard too. This was the first time I spoke at a Tech Conference and I really wanted to do well.

As I said before, I received some really good feedback on the talk and I was really glad to have done it.

Now, you might ask, "Would I have to go to all of this trouble to prepare for a talk?"

Maybe, maybe not. I just happened to find this particular prep process worked well for my brain.

It was nice to hear from some of the attendees surprised that my talk was the first honest to goodness talk on my own I had ever given because it sounded so polished and well done.

Practice makes better, and in this case (based on the videos) it sure did for me

A Big Thank you

A presentation like this took a lot out of me, but I am extremely grateful to a few people in particular:

Katie McLaughlin
The Team of Web Developers at work
Bookie
Chris
Jason
Jon
My daughter Abigail

A little bit extra

If you want to see more details on my talk, here is a playlist of the dry run attempts I did to prepare²

If you want to see the repo where the changes were tracked for my presentation it can be found here

If you want to see my annotated slides, you can find them here

Time tracking

I time track the crap out of my work day, and I really wish I would have done that here just to get a more exact idea of how much time I spent preparing, but some basic back of the envelope math gives me nearly 56 hours of prep for this. One thing I do religiously for work is track my time. I do this for a couple of reasons, but for some reason, I didn't do that as I prepared for my talk here. The time I have here is mostly estimates based on my memory (which could be wildly over stated, or understated).

Activity	Time Spent
Story Teller Tactics Work	1.5 Hours
Mind Mapping Talk	3 hours
Initial Draft of Presentation	5 Hours
Presentation Updates	18 Hours
Deliberate Practice	28 Hours
Total Time	55.5 hours

I use MindNode for mind mapping, mostly on my iPad ↩︎
Be careful, there is at least one not-safe-for-work word in one of the early videos. ↩︎

Error Culture Part III

2023-11-14T00:00:00-08:00

How can I tell if I'm in an error culture?

In part 1 I spoke about the idea of Error Culture. In that post I define what error culture.

In part 2 I spoke when Error Culture starts. This time I'll talk about how you can tell if you're living in an Error Culture, and what you can do about it.

Below are a couple of tell-tale signs I've found to determine if you're living in an error culture.

Email Rules

You start your day and fire up your email client. As the application opens up you see the number of unread message go from 500 down to 20. You think back to a time when you would open your email client and have to trod through ALL 500 of those emails. Now though ... now you've outsmarted the email system by implementing several rules to ignore or hide those pesky emails that don't seem to mean anything.

Instinct to just delete emails

Maybe you don't know about the amazing opportunities that email client rules offer, so you start going through your emails. You delete the ones you know aren't useful or don't mean anything.

Or maybe you do know about rules and of the remaining 20 you notice a few new emails that you don't need to act on. Your first instinct is to delete them, but you remember you are a smart email user and create a new rule to get rid of those emails as well.

Why do I get this email anyway?

If you use rules, you recall a time before you had them. A time when you would methodically read each email and write down a quick note to ask a co-worker, or your boss at your next one on one. But when you brought up the alerts you had one of two reactions:

Oh those ... yeah, you can just delete them. They don't mean anything
Ugh ... how do you not know what that is for? Fine, let me explain it to you ... again

The first item is definitely error culture. The second response could be error culture if the person you've asked is just so overwhelmed with all of the alerts ... OR it could just be a toxic culture. If it's a toxic culture, I'm sorry, but this post might not be helpful in solving that problem.

If you're not in the second situation you may (rightfully) ask

why do we get it if we can just delete it?

And if the answer is 🤷‍♂️ then you might be in an error culture.

In general, if no one knows WHY we're getting an email and there is no actionable direction, you might be in an error culture.

Email Alerts

Ask yourself, your peers, and your boss this question

Is this alert we are getting actually important?

If the answer is No, then delete the mechanism that generates the error. Don't just create a rule to delete the alert.

If the answer is Yes, then ask

Is the alert you are getting actionable?

If the answer is No then update the alert to be actionable. This can be done by

Including steps to resolution or documentation link for resolving the error
Update the alert to indicate it’s importance
Update the alert to go to the correct people

If the answer is Yes then

Make sure the error indicates what the fix needs to be
Make sure the error indicates why it’s important, or a link to documentation that explains it
Make sure the right people are being notified

Point three here is really important. To determine if the correct people are being notified ask this questions of EVERYONE that receives the alert:

Are you the correct person to do something to fix the error

If the answer is No then getting removed from the email is the best course of action.

Of course, it could be that no one ever told you why you were getting the alert so the decision to remove people from alerts may need to be a management level decision, but it can at least start the conversation.

If the answer is Yes then do you (i.e. the person being asked) know what to do to fix the error

Again, with a simple yes or no response, you have two options:

Yes: Does the error indicate what the fix needs to be or where to go to find out? No: Work to update the error to make it actionable

This can help to get the right people getting the alerts.

Below is a flow chart to help make alerts better

None of this is easy to change. You may have managers that don't answer your questions when asking about if someone should receive an alert.

You may not get feedback from your peers, or manager about cleaning up the alert system. But if you can become a champion for the effort it will be very helpful for everyone involved.

If you implement something like this you can increase the signal to noise ratio for you and your team. That seems like a big win for everyone.

Error Culture Part II

2023-11-09T00:00:00-08:00

In my last post I spoke about the idea of Error Culture. In that post I define what error culture. This time I'll talk about when it starts to happen. For a recap go back and read that before diving in here.

When does error culture start?

Error culture can start because of internal reason, external reason, or both and are almost always driven by the best of intentions. Error culture starts to happen because we don't finish the alert process. That is, we set up the alerts, but we don't indicate why they are important or what to do about them when we're notified.

Internal

Internal pressures driving error culture can usually be traced back to someone (usually someone important ¹) declaring that ‘we’ need to be notified of when ‘this’ happens again. In and of itself self, this is actually a really good idea.

But if the important person doesn't identify why we need to be notified all that happens is that an alert is set up and NO ONE knows what to do when it fires off.

The opposite side of the coin here is being proactive in wanting to be notified when a bad thing might happen and being notified might be useful. Again, if there is no definition for why the alert might be useful, you're simply creating noise and encouraging alerts to be ignored.

External

External pressures that can drive error culture are similar to internal ones. There are some slight differences though.

For example, a consultant might indicate that it is best practice TM to be notified of an alert. However, they don't provide more context for why it's best practice. It could very well be that the recommendation IS best practice, but for a user base that is 100x your user base, or for an organization that is 1/10th your size. Context matters and while best practices should scale, they don't always.

Another example of external drivers are software applications provided by third party vendors with default alerts enabled but no context or steps for resolution. Sometimes there will be documentation describing the alert process, but without the context for why the alert is important it's just as likely to be ignored.

So far in this series we've seen what error culture is,and when it starts to happen. In the next post I'll talk about how to identify if you're in an error culture.

important here just means someone with influence ↩︎

Error Culture

2023-10-29T00:00:00-07:00

What is Error Culture?

It's inevitable that at some point a service ¹ will fail. When that service fails you can either choose to be alerted, or not. Because technology is so important to so many aspects of work, not getting an alert for a failing service isn't really an option. So we enable alerts ... for EVERYTHING.

This is good in that we know when things have gone bad ... but it's bad in that we can start to ignore these alerts because we get false positives. If you hear comments like,

Oh yeah, that error always comes up, but we just ignore it because it doesn't mean anything

We don't really know why that error occurs, but it doesn't seem to impact anything, so we just ignore it

This is what I am calling, "Error Culture".

OK, but why is that bad?

Initially, it might not feel bad.

EVERYONE knows that you can ignore that error because it doesn't mean anything. Of course, this knowledge tends to NOT be documented anywhere, so when you onboard new team members they don't know what EVERYONE knows ... because they weren't part of the EVERYONE that learned the lesson.

Additionally, if you're getting error messages and nothing truly bad every happens, then a few things can happen:

People start to question ALL of the alerts. I mean, if this one isn't valid, why is this OTHER one valid? Maybe I can ignore both 🤷‍♂️
You may be getting an alert about a small thing that can be ignored until it's a BIG thing. I think this image does good job of illustrating the point (found here)

Why does it happen?

In general, I've found that error culture can happen for a few reason

Error Fatigue

If you get 1000 alerts every day, you're not going to be able to do anything about anything. This is similar phenomenon to 'Alert Fatgiue' which can happen in software applications (my experience is in Electronic Health Record systems) where users can just click OK or Cancel when an alert shows up and users may not actually see that there is a problem

Lack of understanding of what the error is

It's surprising to find that people that receive alerts and they just delete them. They do this not out malice, but because they honeslty do not know what the alert is for. They were maybe opted into the alert (with no way to opt out) and therefore have no idea why they get it or what they are supposed to do with it. They may also be in an organization where asking questions to learn isn't encouraged and will therefore not ask why they are getting the alert.

Lack of understanding of why the error is important

Related to the item above, but different, a person can receive an alert, but they don't understand why it's important. This is usually manifested in some of the things mentioned before. Ideas like,

well, I've ignored this alert every day for 6 months, I don't know why I need to do anything about it now!

Lack of understand of who the error will impact

I'm reminded of the Episode of Friends where there is a light switch in Chandler and Joey's apartment and they don't know what it's for. At the end of the episide Monica is idly flipping the switch off and on and the camera pans to a Monica and Rachel's apartment where their TV keeps turning off and on.

Error culture can have a similar feeling. If I get an error every few days, but it doesn't impact me or my work I am likely to ignore it. It could be that the error is unimporatnt for me, but HUGELY important for you. This is a case where the error is being directled incorrectly. If we both got the error you could see that I got the email and then ask, hey, are you going to do anything about this?

Emphasis on Hero Culture

This is probably the worst of all possibilities. Some cultures tend to emphasize Heroes or White Knights. They appreciate when someone comes in and 'Saves the Day'. Sometimes people get promoted because of this.

This tends to disincentivize the idea of fixing small problems before they become BIG problems. I might be getting an alert about an issue, but it's not a BIG deal and won't be for some time. Once it becomes a big deal I'll know how to fix it quickly, and I will. When I do, I'll be celebrated. Who wouldn't want that?

In this post I've identified some of the characteristics of Error Culture.

In the next post I'll talk about how to tell if you're in an Error Culture.

In the final post I'll write about what you might be able to do to mitigate, and maybe even eliminate, Error Culture where you are.

When I say service here I mean very loosely anything from a micro service up to a physical server. ↩︎

DjangoCon US 2023

2023-10-24T00:00:00-07:00

My Experience at DjangoCon US 2023

A few days ago I returned from DjangoCon US 2023 and wow, what an amazing time. The only regret I have is that I didn't take very many pictures. This is something I will need to work on for next year.

On Monday October 16th I gave a talk Contributing to Django or how I learned to stop worrying and just try to fix an ORM Bug. The video will be posted on YouTube in a few weeks. This was the first tech conference I've ever spoken at!!!! I was super nervous leading up to the talk, and even a bit at the start, but once I got going I finally settled in.

Here's me on stage taking a selfie with the crowd behind me

Luckily, my talk was one of the first non-Keynote talks so I was able to relax and enjoy the conference while the rest of the time.

After the conference talks ended on Wednesday I stuck around for the sprints. This is such a great time to be able to work on open source projects (Django adjacent or not) and just generally hang out with other Djangonauts. I was able to do some work on DjangoPackages with Jeff Triplett, and just generally hang out with some truly amazing people.

The Django community is just so great. I've been to many conferences before, but this one is the first where I feel like I belong.

I am having some of those post conference blues, but thankfully Kojo Idrissa wrote something about how to help with that. And taking his advice, it has been helpful to come down from the Conference high.

Although the location of DjangoCon US 2024 hasn't been announced yet, I'm making plans to attend.

I am also setting myself some goals to have completed by the start of DCUS 2024

join the fundraising working group
work on at least 1 code related ticket in Trac
work on at least 1 doc related ticket in Trac
have been part of a writing group with fellow Djangonauts and posted at least 1 article per month

I had a great experience speaking, and I think I'd like to do it again, but I'm still working through that.

It's a lot harder to give a talk than I thought it would be! That being said, I do have in my 'To Do' app a task to 'Brainstorm DjangoCon talk ideas' so we'll see if (1) I'm able to come up with anything, and (2) I have a talk accepted for 2024.

Firebirds Inaugural Season

2023-07-01T00:00:00-07:00

On Wednesday June 21, 2023 the local sports puck team (i.e. Hockey), the Coachella Valley Firebirds hosted Game 7 of the Calder Cup Finals against the Hershey Bears.

There are sports writers that can write on how the series went, better than I can so I'll leave that to the pros. What I will talk about is why watching that game and seeing the Firebirds lose in Overtime hit me so hard.

I'm generally an introverted person. Even before the pandemic, I wasn't particularly fond of attending crowded events. The pandemic only intensified my preference for solitude. Suddenly, I found myself being advised to avoid social interactions altogether. As an introvert, the circumstances necessitating isolation weren't exactly ideal for me, but I did appreciate the fact that my family and I had to isolate.

However, after 2+ years of isolating from most everyone, being in large groups would bring out anxiety. And when I say large groups I mean like 10, maybe 15 people. On December 18th there was work holiday get together, the first one since the pandemic started. There were about 100 people in a mostly enclosed space and I did not do well with it. Super anxious, wore a mask the entire time, and generally ducked into the closet that also serves as my office more than once just to get away from people.

That same night was the home opener for the Firebirds at Acrisure Arena (due to construction delays their home arena opened 2 1/2 months after the start of the season). I didn't know it at the time, but it was a sell out (attendance of 10,087). This meant that I was going to a sporting event, in an enclosed arena with 10,000+ people. To say that I nearly lost my shit would be an understatement. The only thing that really got me to go was that the tickets I had purchased weren't cheap, and my wife and I were going with another couple friend.

That first home game was amazing. The Firebirds won 4-3 over the Tucson Roadrunners. The energy was amazing and I decided that I had to go to another game. And so I kept going. Again and again and again. I saw 34 games in person with an average attendance of 7,500.

I'd like to say that "just like that" my anxiety surrounding large indoor gatherings was gone, but it wasn't. It took me going to lots of hockey games to get through it.

So coming back to game 7 on Wednesday night. With less than 1 minute into the second period the Firebirds scored their second goal to go up 2-0. The crowd was the loudest I'd ever heard at Acrisure. Chants of "we want the cup" roared through the arena. It was unreal. And I sat there and realized that if it hadn't been for this team my anxiety surrounding large gatherings wouldn't have gone away for probably a very long time. And other than being a HUGE fan, I wanted the players, coaches, and team to win because they had helped me deal with something so personal. I won't ever be able to repay them for that, but my cheering them on to try and win the cup could maybe start.

And then the unthinkable happened. A penalty was called on the Firebirds and a Power Play goal was scored. Then less than 4 minutes later an even strength goal was scored and we were tied at 2 a piece.

The third period ended without any scorning by either team, and for only the second time in Calder Cup finals history, the first time since 1953, we were going to Overtime in a Game 7.

As we entered Overtime everyone in my section (107) was on their feet. We stood for the entire overtime period. Cheering, and screaming (honestly, I was still exhausted from the experience as I wrote this 2 days later).

About 2 minutes into the Overtime period Ryker Evan sent a shot on goal. From where I was sitting I could see the flight of the puck and my heart leapt as I thought it would find the back of the net ... but sadly it didn't. Within the first five minutes of overtime the Firebirds had outshot the Bears 5-0. It seemed like we were in control.

The next 10 minutes was some of the most intense back and forth hockey I'd ever seen.

With less than 4 minutes on the clock I thought, this might go into double overtime ... and then the unthinkable happened. The Firebirds defense was unable to clear a puck in their end, lots of players in front of the net, and just like that I see a puck flying over Joey's shoulder and past the cross bar, hitting the back of the net. The Bears player and their fans roared with joy, and suddenly a once deafening Acrisure was stunned into silence.

We lost. They won. The inaugural season was over. I stood in disbelief for a minute and then just sat down and stared across the arena at the Bears fans I could see that were losing their minds with joy. I wanted to cry. Some people around me did.

I stood up and looked over at our defensive end. The Firebirds players on the ice had taken a knee as they watched the Bears players celebrate. They don't show that part on TV. The defeated team looking sadly on as the victors celebrate. It was heartbreaking.

And then, in the middle of the celebration, the chants of "Let's go Firebirds" started. In short order, the fans were all saying it as loud as they could. An amazing season that didn't end the way we wanted it to, but we did our best to let the team know what they meant to us.

When I started writing this I thought maybe it was just me that needed something like this to get over some of the anxiety of large indoor gatherings, but maybe it was others. And those others at that game let the team know how much we appreciated them and what they did. This team will always hold a special place in the hearts of it's fans.

We didn't win it all this year, but there's always next year. Always.

Postlude

A friend of a friend of a friend works at a golf course called the 'Classic Club'. There were 3 players that were golfing the next day and they told this friend of a friend of a friend that the chants of "Let's go Firebirds" even after the loss meant so much to them.

GCP Cloud Architect Exam Experience

2023-04-01T00:00:00-07:00

Last October it was announced that Desert Oasis Healthcare (the company I work for) signed on to pilot Google's Care Studio. DOHC is the first ambulatory clinic to sign on.

I had been in some of the discovery meetings before the announcement and was really excited about the opportunity. So far our use of any Cloud platforms at work has been extremely limited (that is to say, we don't use ANY of the big three cloud solutions for our tech) so this seemed to provide a really good opportunity.

As we worked through the project scoping there were conversations about the handoff to DOHC and it occurred to me that I didn't have any knowledge of what GCP offered, what any of it did, or how any of it could work.

I've had on my 'To Do' list to learn one of the Big Three Cloud services (AWS, Azure, or GCP) but because we didn't use ANY of them at work I was (a) worried about picking the 'wrong' one and (b) worried that even if I picked one I'd NEVER be able to use it!

The partnership with Google changed that. Suddenly which cloud service to learn was apparent AND I'd be able to use whatever I learned for work!

Great, now I know which cloud service to start to learn about ... the next question is, "What do I try to learn?". In speaking with some of the folks at Google they recommended one of three Certification options:

After reviewing each of them and having a good idea of what I need to know for work, I opted for the Cloud Architect path.

Knowing which certification I was going to work towards, I started to see what learning options were available for me. It just so happens that Coursera partnered with the California State Library to offer free training which is great because Coursera has a learning path for the Cloud Architect Exam! So I signed up for the first course of that path right before Thanksgiving and started to work my way through the courses.

I spent most of the holidays working through these courses, going pretty fast through them. The labs offered up are so helpful. They actually allow you to work with GCP for FREE during your labs which is amazing.

After I made my way through the Coursera learning Path I bought the book Google Cloud Certified Professional Cloud Architect Study Guide which was really helpful. It came with 100 electronic flash cards and 2 practice exams, and each chapter had questions at the end.

I will say that the practice exams and chapter questions from the book weren't really like the ACTUAL exam questions BUT it did help me in my learning, especially regarding the case studies used in the exams.

I read through the book several times, and used the practice questions in the chapters to drive what parts of the documentation I'd read to shore up my understand of the topics.

Finally, after about 3 months of pretty constant studying I took the test. I opted for the remote proctoring option and I'd say that I really liked this option. I was able to take the test in the same place I had done most of my studying. I did have to remove essentially EVERYTHING from my home office, but not having to drive anywhere, and not having to worry about unfamiliar surroundings really helped me out (I think).

I had 2 hours in which to answer 60 questions. My general strategy for taking tests is to go through the test, mark questions that I'm unsure of and eliminate answers that I know to not be true on those questions. Once I've gone through the test I revisit all of the unsure questions and work through those.

My final pass is to go through ALL of the questions and make sure I didn't do something silly.

Using this strategy I used 1 hour and 50 minutes of the 2 hours ... and I passed!

The unfortunate part of the test is that you only get a Pass or Fail so you don't have any opportunity to know what parts of the exam you missed. Now, if you fail this could be a huge help in working to pass it next time, but even if you pass it I think it would be helpful to know what areas you might struggle in.

All in all this was a pretty great experience and it's already helping with the GCP implementation at work. I'm able to ask better questions because I'm at least aware of the various services and what they do.

Contributing to Django or how I learned to stop worrying and just try to fix an ORM Bug

2022-11-12T00:00:00-08:00

I went to DjangoCon US a few weeks ago and hung around for the sprints. I was particularly interested in working on open tickets related to the ORM. It so happened that Simon Charette was at Django Con and was able to meet with several of us to talk through the inner working of the ORM.

With Simon helping to guide us, I took a stab at an open ticket and settled on 10070. After reviewing it on my own, and then with Simon, it looked like it wasn't really a bug anymore, and so we agreed that I could mark it as done.

Kind of anticlimactic given what I was hoping to achieve, but a closed ticket is a closed ticket! And so I tweeted out my accomplishment for all the world to see.

A few weeks later though, a comment was added that it actually was still a bug and it was reopened.

I was disappointed ... but I now had a chance to actually fix a real bug! I started in earnest.

A suggestion / pattern for working through learning new things that Simon Willison had mentioned was having a public-notes repo on GitHub. He's had some great stuff that he's worked through that you can see here.

Using this as a starting point, I decided to walk through what I learned while working on this open ticket.

Over the course of 10 days I had a 38 comment 'conversation with myself' and it was super helpful!

A couple of key takeaways from working on this issue:

Carlton Gibson said essentially once you start working a ticket from Trac, you are the world's foremost export on that ticket ... and he's right!
... But, you're not working the ticket alone! During the course of my work on the issue I had help from Simon Charette, Mariusz Felisiak, Nick Pope, and Shai Berger
The ORM can seem big and scary ... but remember, it's just Python

I think that each of these lesson learned is important for anyone thinking of contributing to Django (or other open source projects).

That being said, the last point is one that I think can't be emphasized enough.

The ORM has a reputation for being this big black box that only 'really smart people' can understand and contribute to. But, it really is just Python.

If you're using Django, you know (more likely than not) a little bit of Python. Also, if you're using Django, and have written any models, you have a conceptual understanding of what SQL is trying to do (well enough I would argue) that you can get in there AND make sense of what is happening.

And if you know a little bit of Python a great way to learn more is to get into a project like Django and try to fix a bug.

My initial solution isn't the final one that got merged ... it was a collaboration with 4 people, 2 of whom I've never met in real life, and the other 2 I only just met at DjangoCon US a few weeks before.

While working through this I learned just as much from the feedback on my code as I did from trying to solve the problem with my own code.

All of this is to say, contributing to open source can be hard, it can be scary, but honestly, I can't think of a better place to start than Django, and there are lots of places to start.

And for those of you feeling a bit adventurous, there are plenty of ORM tickets just waiting for you to try and fix them!

Upgrading to PostgreSQL 14

2022-08-28T00:00:00-07:00

Django 4.1 was released on August 3, 2022 and I was excited to upgrade to it. I did the testing locally and then pushed my changes up to GitHub to deploy. The deployment was successful, but when I went to visit my sites ... womp womp. I got a Server Error 5XX.

What happened? Well, it turns out that Django 4.1 dropped support for Postgres 10 and that just so happens to be the version I was running on my production server (but not on my local dev machine ... I was running Postgres 14).

OK, so I am going to need to upgrade in order to get the features of anything above Django 4.0 ... and honestly, I've needed to upgrade past Postgres 10 for a while.

I found this StackOverflow question and answer and it helped me a ton! It was to upgrade from Psotgres 10 to 12, but the ideas were the same (but replace 12 with 14). There is also a step that indicates you need to run ./analyze_new_cluster.sh but that seems to be only for version 12(maybe 13) and lower.

Everything was fine until I visited my site and got a Server Error 5XX AGAIN!

What gives?

My first assumption was that maybe the postgres server didn't start back up properly after the upgrade. I checked the service to verify that it was running, and it was

ps -aux | grep postgres

which returned

postgres   988  0.0  1.3 321668 27588 ?        Ss   16:55   0:01 /usr/lib/postgresql/14/bin/postgres -D /var/lib/postgresql/14/main -c config_file=/etc/postgresql/14/main/postgresql.conf
postgres  1034  0.0  0.2 321788  6112 ?        Ss   16:55   0:00 postgres: 14/main: checkpointer
postgres  1035  0.0  0.2 321800  5996 ?        Ss   16:55   0:00 postgres: 14/main: background writer
postgres  1036  0.0  0.4 321668  9388 ?        Ss   16:55   0:00 postgres: 14/main: walwriter
postgres  1039  0.0  0.3 322356  8080 ?        Ss   16:55   0:00 postgres: 14/main: autovacuum launcher
postgres  1040  0.0  0.2 176828  5108 ?        Ss   16:55   0:00 postgres: 14/main: stats collector
postgres  1041  0.0  0.3 322224  6628 ?        Ss   16:55   0:00 postgres: 14/main: logical replication launcher
root      4868  0.0  0.0  14860  1072 pts/0    S+   18:47   0:00 grep --color=auto postgres

I also checked

systemctl status postgresql

which returned as expected

● postgresql.service - PostgreSQL RDBMS
   Loaded: loaded (/lib/systemd/system/postgresql.service; enabled; vendor preset: enabled)
   Active: active (exited) since Sun 2022-08-28 16:55:32 UTC; 1h 54min ago
  Process: 1169 ExecStart=/bin/true (code=exited, status=0/SUCCESS)
 Main PID: 1169 (code=exited, status=0/SUCCESS)

Aug 28 16:55:32 server-name systemd[1]: Starting PostgreSQL RDBMS...
Aug 28 16:55:32 server-name systemd[1]: Started PostgreSQL RDBMS.

One last thing to try

python manage.py makemigrations

This gave me a hint as to what the issue was:

RuntimeWarning: Got an error checking a consistent migration history performed for database connection 'default': connection to server at "127.0.0.1", port 5432 failed: FATAL:  password authentication failed for user "user" connection to server at "127.0.0.1", port 5432 failed: FATAL:

Hmmm ... a quick google search doesn't specifically answer it, but it helps me to get the to answer.

The 'user' isn't able to connect to the database. Maybe the upgrade process resets the password of users in the database or it just doesn't keep the users.

A quick look at the users on the database showed me that the users were still there, so the only thing left to do at this point was to set the user passwords to be what my settings are expecting.

To do that I ran

ALTER USER user WITH PASSWORD 'password';

I did this for the databases that were associated with my websites that were returning 5XX errors and voila! That fixed the issue.

I'm sure that there is a way to keep the passwords for the users after the upgrade, but I haven't been able to find it.

The next time I need to upgrade PostgreSQL I am going to refer back to this post to remind myself what I did last time 😀

A Goodbye to Vin

2022-08-05T00:00:00-07:00

One of the earliest memories of my grandmother is visiting her in 29 Palms ¹ ² in her permanent mobile home. I remember sitting on the davenport watching the Dodgers on a small 13" COLOR CRT TV. I remember that the game was broadcast on KTLA5. But what I remember the most is the voice of Vin Scully.

I don't know what who the Dodgers were playing, but I remember how much my grandmother LOVED to listen to Vin call the game. And it stuck with me. I was probably about 7 or 8 and I thought baseball was "boring". To be fair, I thought most sports were boring, but especially baseball. Nothing ever happens! But, I loved my grandmother, and I loved hanging out with her ³ and so I watched the game with her.

Years later I discovered that yes, I did like baseball, and no, it was not boring. And since my grandmother was a Dodgers fan, then I would be too. It was something that connected us. it didn't matter where I lived, or how old I was, we both loved baseball. We both loved the Dodgers. We both loved to hear Vin call the game.

My grandmother died in 2007, but something that helped to connect me to her in the years since was watching the Dodgers. Listening to Vin.

As Vin got older, he still called the home games, but he handed most of the road games to a new crew. I still loved to Watch Dodgers games, but I loved watching the games he called a little bit more. At the start of each season I always kind of wondered, "is this the last year for Vin?". And in 2016 the answer was yes.

I still remember the last game he called in Dodgers Stadium. I remember the back and forth. I remember the Rockies going up 1 run in the top of the 9th. And the Dodgers tying it back up in the bottom of the 9th. And I remember when Charlie Culberson hit the game winning home run in the bottom of the 10th.

I remember the last game Vin called in San Francisco. I remember the Dodgers lost ... but it was Vin's last game, so I still loved getting the chance to watch it. And to listen to him call the game.

Vin passed at the age of 94 on Aug 2, 2022. Just as I knew that there would be a day when Vin retired from calling games, I knew there would be a day when he wouldn't be with us anymore.

I've been trying process this and figure out why this is hitting me as hard as it is.

It all comes back to my grandmother. They never met each other (at least I don't think they did), but in my head they were inextricably connected. Vin was a connection to my grandmother that I didn't fully realize I had, and with his passing that connection isn't there anymore. He hasn't called a game in more than 5 years, but still, knowing that he NEVER will again is hitting a bit hard for me. And I think it's because it reminds me that my grandma isn't here to watch the games with me anymore, and that bums me out. She was a cool lady who always loved the Dodgers ... and Vin.

WinForVin

Yes that 29 Palms, right next to the LARGEST Marine Corp Base in the WORLD ↩︎
also the 29 Palms that is right next to Joshua Tree home to the National Park that is the current catnip of Hipsters ↩︎
she always had the butter scotch hard candies that were my favorite ↩︎

Django and Legacy Databases

2022-06-15T00:00:00-07:00

I work at a place that is heavily investing in the Microsoft Tech Stack. Windows Servers, c#.Net, Angular, VB.net, Windows Work Stations, Microsoft SQL Server ... etc

When not at work, I really like working with Python and Django. I've never really thought I'd be able to combine the two until I discovered the package mssql-django which was released Feb 18, 2021 in alpha and as a full-fledged version 1 in late July of that same year.

Ever since then I've been trying to figure out how to incorporate Django into my work life.

I'm going to use this series as an outline of how I'm working through the process of getting Django to be useful at work. The issues I run into, and the solutions I'm (hopefully) able to achieve.

I'm also going to use this as a more in depth analysis of an accompanying talk I'm hoping to give at Django Con 2022 later this year.

I'm going to break this down into a several part series that will roughly align with the talk I'm hoping to give. The parts will be:

Introduction/Background
Overview of the Project
Wiring up the Project Models
Database Routers
Django Admin Customization
Admin Documentation
Review & Resources

My intention is to publish one part every week or so. Sometimes the posts will come fast, and other times not. This will mostly be due to how well I'm doing with writing up my findings and/or getting screenshots that will work.

The tool set I'll be using is:

docker
docker-compose
Django
MS SQL
SQLite

Inserting a URL in Markdown in VS Code

2022-04-08T00:00:00-07:00

Since I switched my blog to pelican last summer I've been using VS Code as my writing app. And it's really good for writing, note just code but prose as well.

The one problem I've had is there's no keyboard shortcut for links when writing in markdown ... at least not a default / native keyboard shortcut.

In other (macOS) writing apps you just select the text and press ⌘+k and boop! There's a markdown link set up for you. But not so much in VS Code.

I finally got to the point where that was one thing that may have been keeping me from writing because of how much 'friction' it caused!

So, I decided to figure out how to fix that.

I did have to do a bit of googling and eventually found this StackOverflow answer

Essentially the answer is

Open the Preferences Page: ⌘+Shift+P
Select Preferences: Open Keyboard Shortcuts (JSON)
Update the keybindings.json file to include a new key

The new key looks like this:

{
    "key": "cmd+k",
    "command": "editor.action.insertSnippet",
    "args": {
        "snippet": "[${TM_SELECTED_TEXT}]($0)"
    },
    "when": "editorHasSelection && editorLangId == markdown "
}

Honestly, it's little things like this that can make life so much easier and more fun. Now I just need to remember to do this on my work computer 😀

Logging Part 2

2022-04-07T00:00:00-07:00

In my previous post I wrote about inline logging, that is, using logging in the code without a configuration file of some kind.

In this post I'm going to go over setting up a configuration file to support the various different needs you may have for logging.

Previously I mentioned this scenario:

Perhaps the DevOps team wants robust logging messages on anything ERROR and above, but the application team wants to have INFO and above in a rotating file name schema, while the QA team needs to have the DEBUG and up output to standard out.

Before we get into how we may implement something like what's above, let's review the parts of the Logger which are:

Formatters

In a logging configuration file you can have multiple formatters specified. The above example doesn't state WHAT each team need, so let's define it here:

DevOps: They need to know when the error occurred, what the level was, and what module the error came from
Application Team: They need to know when the error occurred, the level, what module and line
The QA Team: They need to know when the error occurred, the level, what module and line, and they need a stack trace

For the Devops Team we can define a formatter as such¹:

'%(asctime)s - %(levelname)s - %(module)s'

The Application team would have a formatter like this:

'%(asctime)s - %(levelname)s - %(module)s - %(lineno)s'

while the QA team would have one like this:

'%(asctime)s - %(levelname)s - %(module)s - %(lineno)s'

Handlers

The Handler controls where the data from the log is going to be sent. There are several kinds of handlers, but based on our requirements above, we'll only be looking at three of them (see the documentation for more types of handlers)

From the example above we know that the DevOps team wants to save the output to a file, while the Application Team wants to have the log data saved in a way that allows the log files to not get too big. Finally, we know that the QA team wants the output to go directly to stdout

We can handle all of these requirements via the handlers. In this case, we'd use

FileHandler for the DevOps team
RotatingFileHandler for the Application team
StreamHandler for the QA team

Configuration File

Above we defined the formatter and handler. Now we start to put them together. The basic format of a logging configuration has 3 parts (as described above). The example I use below is YAML, but a dictionary or a conf file would also work.

Below we see five keys in our YAML file:

version: 1
formatters:
handlers:
loggers:
root:
  level:
  handlers:

The version key is to allow for future versions in case any are introduced. As of this writing, there is only 1 version ... and it's version: 1

Formatters

We defined the formatters above so let's add them here and give them names that map to the teams

version: 1
formatters:
  devops:
    format: '%(asctime)s - %(levelname)s - %(module)s'
  application:
    format: '%(asctime)s - %(levelname)s - %(module)s - %(lineno)s'
  qa:
    format: '%(asctime)s - %(levelname)s - %(module)s - %(lineno)s'

Right off the bat we can see that the formatters for application and qa are the same, so we can either keep them separate to help allow for easier updates in the future (and to be more explicit) OR we can merge them into a single formatter to adhere to DRY principals.

I'm choosing to go with option 1 and keep them separate.

Handlers

Next, we add our handlers. Again, we give them names to map to the team. There are several keys for the handlers that are specific to the type of handler that is used. For each handler we set a level (which will map to the level from the specs above).

Additionally, each handler has keys associated based on the type of handler selected. For example, logging.FileHandler needs to have the filename specified, while logging.StreamHandler needs to specify where to output to.

When using logging.handlers.RotatingFileHandler we have to specify a few more items in addition to a filename so the logger knows how and when to rotate the log writing.

version: 1
formatters:
  devops:
    format: '%(asctime)s - %(levelname)s - %(module)s'
  application:
    format: '%(asctime)s - %(levelname)s - %(module)s - %(lineno)s'
  qa:
    format: '%(asctime)s - %(levelname)s - %(module)s - %(lineno)s'
handlers:
  devops:
    class: logging.FileHandler
    level: ERROR
    filename: 'devops.log'
  application:
    class: logging.handlers.RotatingFileHandler
    level: INFO
    filename: 'application.log'
    mode: 'a'
    maxBytes: 10000
    backupCount: 3
  qa:
    class: logging.StreamHandler
    level: DEBUG
    stream: ext://sys.stdout

What the setup above does for the devops handler is to output the log data to a file called devops.log, while the application handler outputs to a rotating set of files called application.log. For the application.log it will hold a maximum of 10,000 bytes. Once the file is 'full' it will create a new file called application.log.1, copy the contents of application.log and then clear out the contents of application.log to start over. It will do this 3 times, giving the application team the following files:

application.log
application.log.1
application.log.2

Finally, the handler for QA will output directly to stdout.

Loggers

Now we can take all of the work we did above to create the formatters and handlers and use them in the loggers!

Below we see how the loggers are set up in configuration file. It seems a bit redundant because I've named my formatters, handlers, and loggers all matching terms, but 🤷‍♂️

The only new thing we see in the configuration below is the new propagate: no for each of the loggers. If there were parent loggers (we don't have any) then this would prevent the logging information from being sent 'up' the chain to parent loggers.

The documentation has a good diagram showing the workflow for how the propagate works.

Below we can see what the final, fully formed logging configuration looks like.

version: 1
formatters:
  devops:
    format: '%(asctime)s - %(levelname)s - %(module)s'
  application:
    format: '%(asctime)s - %(levelname)s - %(module)s - %(lineno)s'
  qa:
    format: '%(asctime)s - %(levelname)s - %(module)s - %(lineno)s'
handlers:
  devops:
    class: logging.FileHandler
    level: ERROR
    filename: 'devops.log'
  application:
    class: logging.handlers.RotatingFileHandler
    level: INFO
    filename: 'application.log'
    mode: 'a'
    maxBytes: 10000
    backupCount: 3
  qa:
    class: logging.StreamHandler
    level: DEBUG
    stream: ext://sys.stdout
loggers:
  devops:
    level: ERROR
    formatter: devops
    handlers: [devops]
    propagate: no
  application:
    level: INFO
    formatter: application
    handlers: [application]
    propagate: no
  qa:
    level: DEBUG
    formatter: qa
    handlers: [qa]
    propagate: no
root:
  level: ERROR
  handlers: [devops, application, qa]

In my next post I'll write about how to use the above configuration file to allow the various teams to get the log output they need.

full documentation on what is available for the formatters can be found here: https://docs.python.org/3/library/logging.html#logrecord-attributes ↩︎

Logging Part 1

2022-03-30T00:00:00-07:00

Logging

Last year I worked on an update to the package tryceratops with Gui Latrova to include a verbose flag for logging.

Honestly, Gui was a huge help and I wrote about my experience here but I didn't really understand why what I did worked.

Recently I decided that I wanted to better understand logging so I dove into some posts from Gui, and sat down and read the documentation on the logging from the standard library.

My goal with this was to (1) be able to use logging in my projects, and (2) write something that may be able to help others.

Full disclosure, Gui has a really good article explaining logging and I think everyone should read it. My notes below are a synthesis of his article, my understanding of the documentation from the standard library, and the Python HowTo written in a way to answer the Five W questions I was taught in grade school.

The Five W's

Who are the generated logs for?

Anyone trying to troubleshoot an issue, or monitor the history of actions that have been logged in an application.

What is written to the log?

The formatter determines what to display or store.

When is data written to the log?

The logging level determines when to log the issue.

Where is the log data sent to?

The handler determines where to send the log data whether that's a file, or stdout.

Why would I want to use logging?

To keep a history of actions taken during your code.

How is the data sent to the log?

The loggers determine how to bundle all of it together through calls to various methods.

Examples

Let's say I want a logger called my_app_errors that captures all ERROR level incidents and higher to a file and to tell me the date time, level, message, logger name, and give a trace back of the error, I could do the following:

import logging

message='oh no! an error occurred'
formatter = logging.Formatter('%(asctime)s - %(levelname)s - %(message)s - %(name)s')
logger = logging.getLogger('my_app_errors')
fh = logging.FileHandler('errors.log')
fh.setFormatter(formatter)
logger.addHandler(fh)
logger.error(message, stack_info=True)

The code above would generate something like this to a file called errors.log

2022-03-28 19:45:49,188 - ERROR - oh no! an error occurred - my_app_errors
Stack (most recent call last):
  File "/Users/ryan/Documents/github/logging/test.py", line 9, in <module>
    logger.error(message, stack_info=True)

If I want a logger that will do all of the above AND output debug information to the console I could:

import logging

message='oh no! an error occurred'

logger = logging.getLogger('my_app_errors')

ch = logging.StreamHandler()
fh = logging.FileHandler('errors.log')

formatter = logging.Formatter('%(asctime)s - %(levelname)s - %(message)s - %(name)s')

fh.setFormatter(formatter)
ch.setFormatter(formatter)

logger.addHandler(fh)
logger.addHandler(ch)

logger.error(message, stack_info=True)
logger.debug(message, stack_info=True)

Again, the code above would generate something like this to a file called errors.log

2022-03-28 19:45:09,406 - ERROR - oh no! an error occurred - my_app_errors
Stack (most recent call last):
  File "/Users/ryan/Documents/github/logging/test.py", line 18, in <module>
    logger.error(message, stack_info=True)

but it would also output to stderr in the terminal something like this:

2022-03-27 13:18:45,367 - ERROR - oh no! an error occurred - my_app_errors
Stack (most recent call last):
  File "<stdin>", line 1, in <module>

The above it a bit hard to scale though. What happens when we want to have multiple formatters, for different levels that get output to different places? We can incorporate all of that into something like what we see above, OR, we can stat to leverage the use of logging configuration files.

Why would we want to have multiple formatters? Perhaps the DevOps team wants robust logging messages on anything ERROR and above, but the application team wants to have INFO and above in a rotating file name schema, while the QA team needs to have the DEBUG and up output to standard out.

You CAN do all of this inline with the code above, but would you really want to? Probably not.

Enter configuration files to allow easier management of log files (and a potential way to make everyone happy) which I'll cover in the next post.

New Theme, who dis?

2022-02-27T00:00:00-08:00

Because I have a couple of posts that I need/want to work on, and I have the time to work on them, I have of course decided to instead to update the theme on my blog because that was a way better use of my time 😂

Also, because the day is just too nice to not be sitting outside watching baseball (even if it's on TV ... and even if it's the ping of the bat and not the crack of the bat¹)

Since the MLB Lockout is still going on and there's no end in sight, I've resorted to watching NCAA Baseball. I have to say, it's really entertaining AND it seems like there's 100 games on each day! ↩︎

I made a Slackbot!

2022-02-19T00:00:00-08:00

Building my first Slack Bot

I had added a project to my OmniFocus database in November of 2021 which was, "Build a Slackbot" after watching a Video by Mason Egger. I had hoped that I would be able to spend some time on it over the holidays, but I was never able to really find the time.

A few weeks ago, Bob Belderbos tweeted:

If you were to build a Slack bot, what would it do?
— Bob Belderbos (@bbelderbos) February 2, 2022

And I responded

I work in US Healthcare where there are a lot of Acronyms (many of which are used in tech but have different meaning), so my slack bot would allow a user to enter an acronym and return what it means, i.e., CMS = Centers for Medicare and Medicaid Services.
— The B Is Silent (@ryancheley) February 2, 2022

I didn't really have anymore time now than I did over the holiday, but Bob asking and me answering pushed me to actually write the darned thing.

I think one of the problems I encountered was what backend / tech stack to use. I'm familiar with Django, but going from 0 to something in production has a few steps and although I know how to do them ... I just felt ~overwhelmed~ by the prospect.

I felt equally ~overwhelmed~ by the prospect of trying FastAPI to create the API or Flask, because I am not as familiar with their deployment story.

Another thing that was different now than before was that I had worked on a Django Cookie Cutter to use and that was 'good enough' to try it out. So I did.

I ran into a few problems while working with my Django Cookie Cutter but I fixed them and then dove head first into writing the Slack Bot

The model

The initial implementation of the model was very simple ... just 2 fields:

class Acronym(models.Model):
    acronym = models.CharField(max_length=8)
    definition = models.TextField()

    def save(self, *args, **kwargs):
        self.acronym = self.acronym.lower()
        super(Acronym, self).save(*args, **kwargs)

    class Meta:
        unique_together = ("acronym", "definition")
        ordering = ["acronym"]

    def __str__(self) -> str:
        return self.acronym

Next I created the API using Django Rest Framework using a single serializer

class AcronymSerializer(serializers.ModelSerializer):
    class Meta:
        model = Acronym
        fields = [
            "id",
            "acronym",
            "definition",
        ]

which is used by a single view

class AcronymViewSet(viewsets.ReadOnlyModelViewSet):
    serializer_class = AcronymSerializer
    queryset = Acronym.objects.all()

    def get_object(self):
        queryset = self.filter_queryset(self.get_queryset())
        print(self.kwargs["acronym"])
        acronym = self.kwargs["acronym"]
        obj = get_object_or_404(queryset, acronym__iexact=acronym)

        return obj

and exposed on 2 end points:

from django.urls import include, path

from .views import AcronymViewSet, AddAcronym, CountAcronyms, Events

app_name = "api"

user_list = AcronymViewSet.as_view({"get": "list"})
user_detail = AcronymViewSet.as_view({"get": "retrieve"})

urlpatterns = [
    path("", AcronymViewSet.as_view({"get": "list"}), name="acronym-list"),
    path("<acronym>/", AcronymViewSet.as_view({"get": "retrieve"}), name="acronym-detail"),
    path("api-auth/", include("rest_framework.urls", namespace="rest_framework")),
]

Getting the data

At my joby-job we use Jira and Confluence. In one of our Confluence spaces we have a Glossary page which includes nearly 200 acronyms. I had two choices:

Copy and Paste the acronym and definition for each item
Use Python to get the data

I used Python to get the data, via a Jupyter Notebook, but I didn't seem to save the code anywhere (🤦🏻), so I can't include it here. But trust me, it was 💯.

Setting up the Slack Bot

Although I had watched Mason's video, since I was building this with Django I used this article as a guide in the development of the code below.

The code from my views.py is below:

ssl_context = ssl.create_default_context()
ssl_context.check_hostname = False
ssl_context.verify_mode = ssl.CERT_NONE

SLACK_VERIFICATION_TOKEN = getattr(settings, "SLACK_VERIFICATION_TOKEN", None)
SLACK_BOT_USER_TOKEN = getattr(settings, "SLACK_BOT_USER_TOKEN", None)
CONFLUENCE_LINK = getattr(settings, "CONFLUENCE_LINK", None)
client = slack.WebClient(SLACK_BOT_USER_TOKEN, ssl=ssl_context)

class Events(APIView):
    def post(self, request, *args, **kwargs):

        slack_message = request.data

        if slack_message.get("token") != SLACK_VERIFICATION_TOKEN:
            return Response(status=status.HTTP_403_FORBIDDEN)

        # verification challenge
        if slack_message.get("type") == "url_verification":
            return Response(data=slack_message, status=status.HTTP_200_OK)
        # greet bot
        if "event" in slack_message:
            event_message = slack_message.get("event")

            # ignore bot's own message
            if event_message.get("subtype"):
                return Response(status=status.HTTP_200_OK)

            # process user's message
            user = event_message.get("user")
            text = event_message.get("text")
            channel = event_message.get("channel")
            url = f"https://slackbot.ryancheley.com/api/{text}/"
            response = requests.get(url).json()
            definition = response.get("definition")
            if definition:
                message = f"The acronym '{text.upper()}' means: {definition}"
            else:
                confluence = CONFLUENCE_LINK + f'/dosearchsite.action?cql=siteSearch+~+"{text}"'
                confluence_link = f"<{confluence}|Confluence>"
                message = f"I'm sorry <@{user}> I don't know what *{text.upper()}* is :shrug:. Try checking {confluence_link}."

            if user != "U031T0UHLH1":
                client.chat_postMessage(
                    blocks=[{"type": "section", "text": {"type": "mrkdwn", "text": message}}], channel=channel
                )
                return Response(status=status.HTTP_200_OK)
        return Response(status=status.HTTP_200_OK)

Essentially what the Slack Bot does is takes in the request.data['text'] and checks it against the DRF API end point to see if there is a matching Acronym.

If there is, then it returns the acronym and it's definition.

If it's not, you get a message that it's not sure what you're looking for, but that maybe Confluence¹ can help, and gives a link to our Confluence Search page.

The last thing you'll notice is that if the User has a specific ID it won't respond with a message. That's because in my initial testing I just had the Slack Bot replying to the user saying 'Hi' with a 'Hi' back to the user.

I had a missing bit of logic though, so once you said hi to the Slack Bot, it would reply back 'Hi' and then keep replying 'Hi' because it was talking to itself. It was comical to see in real time 😂.

Using ngrok to test it locally

ngrok is a great tool for taking a local url, like localhost:8000/api/entpoint, and exposing it on the internet with a url like https://a123-45-678-901-234.ngrok.io/api/entpoint. This allows you to test your local code and see any issues that might arise when pushed to production.

As I mentioned above the Slack Bot continually said "Hi" to itself in my initial testing. Since I was running ngrok to serve up my local Server I was able to stop the infinite loop by stopping my local web server. This would have been a little more challenging if I had to push my code to an actual web server first and then tested.

Conclusion

This was such a fun project to work on, and I'm really glad that Bob tweeted asking what Slack Bot we would build.

That gave me the final push to actually build it.

You'll notice that I'm using an environment variable to define the Confluence Link and may wonder why. It's mostly to keep the actual Confluence Link used at work non-public and not for any other reason 🤷🏻 ↩︎

Putting it All Together

2022-02-09T00:00:00-08:00

In this final post I'll be writing up how everything fits together. As a recap, here are the steps I go through to create and publish a new post

Create Post

Create .md for my new post
write my words
edit post
Change status from draft to published

Publish Post

Run make html to generate the SQLite database that powers my site's search tool¹
Run make vercel to deploy the SQLite database to vercel
Run git add <filename> to add post to be committed to GitHub
Run git commit -m <message> to commit to GitHub
Post to Twitter with a link to my new post

My previous posts have gone over how each step was automated, but now we'll 'throw it all together'.

I updated my Makefile with a new command:

tweet:
    ./tweet.sh

When I run make tweet it will calls tweet.sh. I wrote about the tweet.sh file in Auto Generating the Commit Message so I won't go deeply into here. What it does is automate steps 1 - 5 above for the Publish Post section above.

And that's it really. I've now been able to automate the file creation and publish process.

Admittedly these are the 'easy' parts. The hard part is the actual writing, but it does remove a ton pf potential friction from my workflow and this will hopefully lead to more writing this year.

make vercel actually runs make html so this isn't really a step that I need to do. ↩︎

Automating the file creation

2022-02-02T00:00:00-08:00

In my last post Auto Generating the Commit Message I indicated that this post I would "throw it all together and to get a spot where I can run one make command that will do all of this for me".

I decided to take a brief detour though as I realized I didn't have a good way to create a new post, i.e. the starting point wasn't automated!

In this post I'm going to go over how I create the start to a new post using Makefile and the command make newpost

My initial idea was to create a new bash script (similar to the tweet.sh file), but as a first iteration I went in a different direction based on this post How to Slugify Strings in Bash.

The command that the is finally arrived at in the post above was

newpost:
    vim +':r templates/post.md' $(BASEDIR)/content/blog/$$(date +%Y-%m-%d)-$$(echo -n $${title} | sed -e 's/[^[:alnum:]]/-/g' | tr -s '-' | tr A-Z a-z.md).md

which was really close to what I needed. My static site is set up a bit differently and I'm not using vim (I'm using VS Code) to write my words.

The first change I needed to make was to remove the use of vim from the command and instead use touch to create the file

newpost:
    touch $(BASEDIR)/content/blog/$$(date +%Y-%m-%d)-$$(echo -n $${title} | sed -e 's/[^[:alnum:]]/-/g' | tr -s '-' | tr A-Z a-z.md).md

The second was to change the file path for where to create the file. As I've indicated previously, the structure of my content looks like this:

content
├── musings
├── pages
├── productivity
├── professional\ development
└── technology

giving me an updated version of the command that looks like this:

touch content/$$(echo $${category})/$$(echo $${title} | sed -e 's/[^[:alnum:]]/-/g' | tr -s '-' | tr A-Z a-z.md).md

When I run the command make newpost title='Automating the file creation' category='productivity' I get a empty new files created.

Now I just need to populate it with the data.

There are seven bits of meta data that need to be added, but four of them are the same for each post

Author: ryan
Tags:
Series: Remove if Not Needed
Status: draft

That allows me to have the newpost command look like this:

newpost:
    touch content/$$(echo $${category})/$$(echo $${title} | sed -e 's/[^[:alnum:]]/-/g' | tr -s '-' | tr A-Z a-z.md).md
    echo "Author: ryan" >> content/$$(echo $${category})/$$(echo $${title} | sed -e 's/[^[:alnum:]]/-/g' | tr -s '-' | tr A-Z a-z.md).md
    echo "Tags: " >> content/$$(echo $${category})/$$(echo $${title} | sed -e 's/[^[:alnum:]]/-/g' | tr -s '-' | tr A-Z a-z.md).md
    echo "Series: Remove if Not Needed"  >> content/$$(echo $${category})/$$(echo $${title} | sed -e 's/[^[:alnum:]]/-/g' | tr -s '-' | tr A-Z a-z.md).md
    echo "Status: draft"  >> content/$$(echo $${category})/$$(echo $${title} | sed -e 's/[^[:alnum:]]/-/g' | tr -s '-' | tr A-Z a-z.md).md

The remaining metadata to be added are:

Title:
Date
Slug

Of these, Date and Title are the most straightforward.

bash has a command called date that can be formatted in the way I want with %F. Using this I can get the date like this

echo "Date: $$(date +%F)" >> content/$$(echo $${category})/$$(echo $${title} | sed -e 's/[^[:alnum:]]/-/g' | tr -s '-' | tr A-Z a-z.md).md

For Title I can take the input parameter title like this:

echo "Title: $${title}" > content/$$(echo $${category})/$$(echo $${title} | sed -e 's/[^[:alnum:]]/-/g' | tr -s '-' | tr A-Z a-z.md).md

Slug is just Title but slugified. Trying to figure out how to do this is how I found the article above.

Using a slightly modified version of the code that generates the file, we get this:

printf "Slug: " >> content/$$(echo $${category})/$$(echo $${title} | sed -e 's/[^[:alnum:]]/-/g' | tr -s '-' | tr A-Z a-z.md).md
echo "$${title}" | sed -e 's/[^[:alnum:]]/-/g' | tr -s '-' >> content/$$(echo $${category})/$$(echo $${title} | sed -e 's/[^[:alnum:]]/-/g' | tr -s '-' | tr A-Z a-z.md).md

One thing to notice here is that printf. I needed/wanted to echo -n but make didn't seem to like that. This StackOverflow answer helped me to get a fix (using printf) though I'm sure there's a way I can get it to work with echo -n.

Essentially, since this was a first pass, and I'm pretty sure I'm going to end up re-writing this as a shell script I didn't want to spend too much time getting a perfect answer here.

OK, with all of that, here's the entire newpost recipe I'm using now:

newpost:
    touch content/$$(echo $${category})/$$(echo $${title} | sed -e 's/[^[:alnum:]]/-/g' | tr -s '-' | tr A-Z a-z.md).md
    echo "Title: $${title}" > content/$$(echo $${category})/$$(echo $${title} | sed -e 's/[^[:alnum:]]/-/g' | tr -s '-' | tr A-Z a-z.md).md
    echo "Date: $$(date +%F)" >> content/$$(echo $${category})/$$(echo $${title} | sed -e 's/[^[:alnum:]]/-/g' | tr -s '-' | tr A-Z a-z.md).md
    echo "Author: ryan" >> content/$$(echo $${category})/$$(echo $${title} | sed -e 's/[^[:alnum:]]/-/g' | tr -s '-' | tr A-Z a-z.md).md
    echo "Tags: " >> content/$$(echo $${category})/$$(echo $${title} | sed -e 's/[^[:alnum:]]/-/g' | tr -s '-' | tr A-Z a-z.md).md
    printf "Slug: " >> content/$$(echo $${category})/$$(echo $${title} | sed -e 's/[^[:alnum:]]/-/g' | tr -s '-' | tr A-Z a-z.md).md
    echo "$${title}" | sed -e 's/[^[:alnum:]]/-/g' | tr -s '-' | tr A-Z a-z >> content/$$(echo $${category})/$$(echo $${title} | sed -e 's/[^[:alnum:]]/-/g' | tr -s '-' | tr A-Z a-z.md).md
    echo "Series: Remove if Not Needed"  >> content/$$(echo $${category})/$$(echo $${title} | sed -e 's/[^[:alnum:]]/-/g' | tr -s '-' | tr A-Z a-z.md).md
    echo "Status: draft"  >> content/$$(echo $${category})/$$(echo $${title} | sed -e 's/[^[:alnum:]]/-/g' | tr -s '-' | tr A-Z a-z.md).md

This allows me to type make newpost and generate a new file for me to start my new post in!¹

When this post was originally published the slug command didn't account for making all of the text lower case. This was fixed in a subsequent commit ↩︎

Auto Generating the Commit Message

2022-01-28T00:00:00-08:00

In my first post of this series I outlined the steps needed in order for me to post. They are:

Run make html to generate the SQLite database that powers my site's search tool¹
Run make vercel to deploy the SQLite database to vercel
Run git add <filename> to add post to be committed to GitHub
Run git commit -m <message> to commit to GitHub
Post to Twitter with a link to my new post

In this post I'll be focusing on how I automated step 4, Run git commit -m <message> to commit to GitHub.

Automating the "git commit ..." part of my workflow

In order for my GitHub Action to auto post to Twitter, my commit message needs to be in the form of "New Post: ...". What I'm looking for is to be able to have the commit message be something like this:

New Post: Great New Post https://ryancheley.com/yyyy/mm/dd/great-new-post/

This is basically just three parts from the markdown file, the Title, the Date, and the Slug.

In order to get those details, I need to review the structure of the markdown file. For Pelican writing in markdown my file is structured like this:

Title:
Date:
Tags:
Slug:
Series:
Authors:
Status:

My words start here and go on for a bit.

In the last post I wrote about how to git add the files in the content directory. Here, I want to take the file that was added to git and get the first 7 rows, i.e. the details from Title to Status.

The file that was updated that needs to be added to git can be identified by running

find content -name '*.md' -print | sed 's/^/"/g' | sed 's/$/"/g' | xargs git add

Running git status now will display which file was added with the last command and you'll see something like this:

❯ git status
On branch main
Untracked files:
  (use "git add <file>..." to include in what will be committed)
        content/productivity/auto-generating-the-commit-message.md

What I need though is a more easily parsable output. Enter the porcelin flag which, per the docs

Give the output in an easy-to-parse format for scripts. This is similar to the short output, but will remain stable across Git versions and regardless of user configuration. See below for details.

which is exactly what I needed.

Running git status --porcelain you get this:

❯ git status --porcelain
?? content/productivity/more-writing-automation.md

Now, I just need to get the file path and exclude the status (the ?? above in this case²), which I can by piping in the results and using sed

❯ git status --porcelain | sed s/^...//
content/productivity/more-writing-automation.md

The sed portion says

search the output string starting at the beginning of the line (^)
find the first three characters (...). ³
replace them with nothing (//)

There are a couple of lines here that I need to get the content of for my commit message:

Title
Slug
Date
Status⁴

I can use head to get the first n lines of a file. In this case, I need the first 7 lines of the output from git status --porcelain | sed s/^...//. To do that, I pipe it to head!

git status --porcelain | sed s/^...// | xargs head -7

That command will return this:

Title: Auto Generating the Commit Message
Date: 2022-01-24
Tags: Automation
Slug: auto-generating-the-commit-message
Series: Auto Deploying my Words
Authors: ryan
Status: draft

In order to get the Title, I'll pipe this output to grep to find the line with Title

git status --porcelain | sed s/^...// | xargs head -7 | grep 'Title: '

which will return this

Title: Auto Generating the Commit Message

Now I just need to remove the leading Title: and I've got the title I'm going to need for my Commit message!

git status --porcelain | sed s/^...// | xargs head -7 | grep 'Title: ' | sed -e 's/Title: //g'

which return just

Auto Generating the Commit Message

I do this for each of the parts I need:

Title
Slug
Date
Status

Now, this is getting to have a lot of parts, so I'm going to throw it into a bash script file called tweet.sh. The contents of the file look like this:

TITLE=`git status --porcelain | sed s/^...// | xargs head -7 | grep 'Title: ' | sed -e 's/Title: //g'`
SLUG=`git status --porcelain | sed s/^...// | xargs head -7 | grep 'Slug: ' | sed -e 's/Slug: //g'`
POST_DATE=`git status --porcelain | sed s/^...// | xargs head -7 | grep 'Date: ' | sed -e 's/Date: //g' | head -c 10 | grep '-' | sed -e 's/-/\//g'`
POST_STATUS=` git status --porcelain | sed s/^...// | xargs head -7 | grep 'Status: ' | sed -e 's/Status: //g'`

You'll see above that the Date piece is a little more complicated, but it's just doing a find and replace on the - to update them to / for the URL.

Now that I've got all of the pieces I need, it's time to start putting them together

I define a new variable called URL and set it

URL="https://ryancheley.com/$POST_DATE/$SLUG/"

and the commit message

MESSAGE="New Post: $TITLE $URL"

Now, all I need to do is wrap this in an if statement so the command only runs when the STATUS is published

if [ $POST_STATUS = "published" ]
then
    MESSAGE="New Post: $TITLE $URL"

    git commit -m "$MESSAGE"

    git push github main
fi

Putting this all together (including the git add from my previous post) and the tweet.sh file looks like this:

# Add the post to git
find content -name '*.md' -print | sed 's/^/"/g' | sed 's/$/"/g' | xargs git add


# Get the parts needed for the commit message
TITLE=`git status --porcelain | sed s/^...// | xargs head -7 | grep 'Title: ' | sed -e 's/Title: //g'`
SLUG=`git status --porcelain | sed s/^...// | xargs head -7 | grep 'Slug: ' | sed -e 's/Slug: //g'`
POST_DATE=`git status --porcelain | sed s/^...// | xargs head -7 | grep 'Date: ' | sed -e 's/Date: //g' | head -c 10 | grep '-' | sed -e 's/-/\//g'`
POST_STATUS=` git status --porcelain | sed s/^...// | xargs head -7 | grep 'Status: ' | sed -e 's/Status: //g'`

URL="https://ryancheley.com/$POST_DATE/$SLUG/"

if [ $POST_STATUS = "published" ]
then
    MESSAGE="New Post: $TITLE $URL"

    git commit -m "$MESSAGE"

    git push github main
fi

When this script is run it will find an updated or added markdown file (i.e. article) and add it to git. It will then parse the file to get data about the article. If the article is set to published it will commit the file with a message and will push to github. Once at GitHub, the Tweeting action I wrote about will tweet my commit message!

In the next (and last) article, I'm going to throw it all together and to get a spot when I can run one make command that will do all of this for me.

Caveats

The script above works, but if you have multiple articles that you're working on at the same time, it will fail pretty spectacularly. The final version of the script has guards against that and looks like this

make vercel actually runs make html so this isn't really a step that I need to do. ↩︎
Other values could just as easily be M or A ↩︎
Why the first three characters, because that's how porcelain outputs the status ↩︎
I will also need the Status to do some conditional logic otherwise I may have a post that is in draft status that I want to commit and the GitHub Action will run posting a tweet with an article and URL that don't actually exist yet. ↩︎

git add filename automation

2022-01-26T00:00:00-08:00

In my last post I mentioned the steps needed in order for me to post. They are:

Run make html to generate the SQLite database that powers my site's search tool¹
Run make vercel to deploy the SQLite database to vercel
Run git add <filename> to add post to be committed to GitHub
Run git commit -m <message> to commit to GitHub
Post to Twitter with a link to my new post

In that post I focused on number 5, posting to Twitter with a link to the post using GitHub Actions.

In this post I'll be focusing on how I automated step 3, "Run git add <filename> to add post to be committed to GitHub".

Automating the `git add ...` part of my workflow

I have my pelican content set up so that the category of a post is determined by the directory a markdown file is placed in. The structure of my content folder looks like this:

content
├── musings
├── pages
├── productivity
├── professional\ development
└── technology

If you just just git status on a directory it will give you the status of all of the files in that directory that have been changed, added, removed. Something like this:

❯ git status
On branch main
Untracked files:
  (use "git add <file>..." to include in what will be committed)
        content/productivity/more-writing-automation.md
        Makefile
        metadata.json

That means that when you run git add . all of those files will be added to git. For my purposes all that I need is the one updated file in the content directory.

The command find does a great job of taking a directory and allowing you to search for what you want in that directory. You can run something like

find content -name '*.md' -print

And it will return essentially what you're looking for. Something like this:

content/pages/404.md
content/pages/curriculum-vitae.md
content/pages/about.md
content/pages/brag.md
content/productivity/adding-the-new-file.md
content/productivity/omnifocus-3.md
content/productivity/making-the-right-choice-or-how-i-learned-to-live-with-limiting-my-own-technical-debt-and-just-be-happy.md
content/productivity/auto-tweeting-new-post.md
content/productivity/my-outlook-review-process.md
content/productivity/rules-and-actions-in-outlook.md
content/productivity/auto-generating-the-commit-message.md
content/productivity/declaring-omnifocus-bankrupty.md

However, because one of my categories has a space in it's name (professional development) if you pipe the output of this to xargs git add it fails with the error

fatal: pathspec 'content/professional' did not match any files

In order to get around this, you need to surround the output of the results of find with double quotes ("). You can do this by using sed

find content -name '*.md' -print | sed 's/^/"/g' | sed 's/$/"/g'

What this says is, take the output of find and pipe it to sed and use a global find and replace to add a " to the start of the line (that's what the ^ does) and then pipe that to sed again and use a global find and replace to add a " to the end of the line (that's what the '$' does).

Now, when you run

find content -name '*.md' -print | sed 's/^/"/g' | sed 's/$/"/g'

The output looks like this:

"content/pages/404.md"
"content/pages/curriculum-vitae.md"
"content/pages/about.md"
"content/pages/brag.md"
"content/productivity/adding-the-new-file.md"
"content/productivity/omnifocus-3.md"
"content/productivity/making-the-right-choice-or-how-i-learned-to-live-with-limiting-my-own-technical-debt-and-just-be-happy.md"
"content/productivity/auto-tweeting-new-post.md"
"content/productivity/my-outlook-review-process.md"
"content/productivity/rules-and-actions-in-outlook.md"
"content/productivity/auto-generating-the-commit-message.md"
"content/productivity/declaring-omnifocus-bankrupty.md"

Now, you can pipe your output to xargs git add and there is no error!

The final command looks like this:

find content -name '*.md' -print | sed 's/^/"/g' | sed 's/$/"/g' | xargs git add

In the next post, I'll walk through how I generate the commit message to be used in the automatic tweet!

make vercel actually runs make html so this isn't really a step that I need to do. ↩︎

Auto Tweeting New Post

2022-01-24T00:00:00-08:00

Each time I write something for this site there are several steps that I go through to make sure that the post makes it's way to where people can see it.

Run make html to generate the SQLite database that powers my site's search tool¹
Run make vercel to deploy the SQLite database to vercel
Run git add <filename> to add post to be committed to GitHub
Run git commit -m <message> to commit to GitHub
Post to Twitter with a link to my new post

If there's more than 2 things to do, I'm totally going to forget to do one of them.

The above steps are all automat-able, but the one I wanted to tackle first was the automated tweet. Last night I figured out how to tweet with a GitHub action.

There were a few things to do to get the auto tweet to work:

Find a GitHub in the Market Place that did the auto tweet (or try to write one if I couldn't find one)
Set up a twitter app with Read and Write privileges
Set the necessary secrets for the report (API Key, API Key Secret, Access Token, Access Token Secret, Bearer)
Test the GitHub Action

The action I chose was send-tweet-action. It's got easy to read documentation on what is needed. Honestly the hardest part was getting a twitter app set up with Read and Write privileges.

I'm still not sure how to do it, honestly. I was lucky enough that I already had an app sitting around with Read and Write from the WordPress blog I had previously, so I just regenerated the keys for that one and used them.

The last bit was just testing the action and seeing that it worked as expected. It was pretty cool running an action and then seeing a tweet in my timeline.

The TIL for this was that GitHub Actions can have conditionals. This is important because I don't want to generate a new tweet each time I commit to main. I only want that to happen when I have a new post.

To do that, you just need this in the GitHub Action:

    if: "contains(github.event.head_commit.message, '<String to Filter on>')"

In my case, the <String to Filter on> is New Post:.

The send-tweet-action has a status field which is the text tweeted. I can use the github.event.head_commit.message in the action like this:

    ${{ github.event.head_commit.message }}

Now when I have a commit message that starts 'New Post:' against main I'll have a tweet get sent out too!

This got me to thinking that I can/should automate all of these steps.

With that in mind, I'm going to work on getting the process down to just having to run a single command. Something like:

    make publish "New Post: Title of my Post https://www.ryancheley.com/yyyy/mm/dd/slug/"

make vercel actually runs make html so this isn't really a step that I need to do. ↩︎

Adding Search to My Pelican Blog with Datasette

2022-01-16T19:30:00-08:00

Last summer I migrated my blog from Wordpress to Pelican. I did this for a couple of reasons (see my post here), but one thing that I was a bit worried about when I migrated was that Pelican's offering for site search didn't look promising.

There was an outdated plugin called tipue-search but when I was looking at it I could tell it was on it's last legs.

I thought about it, and since my blag isn't super high trafficked AND you can use google to search a specific site, I could wait a bit and see what options came up.

After waiting a few months, I decided it would be interesting to see if I could write a SQLite utility to get the data from my blog, add it to a SQLite database and then use datasette to serve it up.

I wrote the beginning scaffolding for it last August in a utility called pelican-to-sqlite, but I ran into several technical issues I just couldn't overcome. I thought about giving up, but sometimes you just need to take a step away from a thing, right?

After the first of the year I decided to revisit my idea, but first looked to see if there was anything new for Pelican search. I found a tool plugin called search that was released last November and is actively being developed, but as I read through the documentation there was just A LOT of stuff:

stork
requirements for the structure of your page html
static asset hosting
deployment requires updating your nginx settings

These all looked a bit scary to me, and since I've done some work using datasette I thought I'd revisit my initial idea.

My First Attempt

As I mentioned above, I wrote the beginning scaffolding late last summer. In my first attempt I tried to use a few tools to read the md files and parse their yaml structure and it just didn't work out. I also realized that Pelican can have reStructured Text and that any attempt to parse just the md file would never work for those file types.

My Second Attempt

The Plugin

During the holiday I thought a bit about approaching the problem from a different perspective. My initial idea was to try and write a datasette style package to read the data from pelican. I decided instead to see if I could write a pelican plugin to get the data and then add it to a SQLite database. It turns out, I can, and it's not that hard.

Pelican uses signals to make plugin in creation a pretty easy thing. I read a post and the documentation and was able to start my effort to refactor pelican-to-sqlite.

From The missing Pelican plugins guide I saw lots of different options, but realized that the signal article_generator_write_article is what I needed to get the article content that I needed.

I then also used sqlite_utils to insert the data into a database table.

def save_items(record: dict, table: str, db: sqlite_utils.Database) -> None:  # pragma: no cover
    db[table].insert(record, pk="slug", alter=True, replace=True)

Below is the method I wrote to take the content and turn it into a dictionary which can be used in the save_items method above.

def create_record(content) -> dict:
    record = {}
    author = content.author.name
    category = content.category.name
    post_content = html2text.html2text(content.content)
    published_date = content.date.strftime("%Y-%m-%d")
    slug = content.slug
    summary = html2text.html2text(content.summary)
    title = content.title
    url = "https://www.ryancheley.com/" + content.url
    status = content.status
    if status == "published":
        record = {
            "author": author,
            "category": category,
            "content": post_content,
            "published_date": published_date,
            "slug": slug,
            "summary": summary,
            "title": title,
            "url": url,
        }
    return record

Putting these together I get a method used by the Pelican Plugin system that will generate the data I need for the site AND insert it into a SQLite database

def run(_, content):
    record = create_record(content)
    save_items(record, "content", db)

def register():
    signals.article_generator_write_article.connect(run)

The html template update

I use a custom implementation of Smashing Magazine. This allows me to do some edits, though I mostly keep it pretty stock. However, this allowed me to make a small edit to the base.html template to include a search form.

In order to add the search form I added the following code to base.html below the nav tag:

    <section class="relative h-8">
    <section class="absolute inset-y-0 right-10 w-128">
    <form
    class = "pl-4"
    <
    action="https://search-ryancheley.vercel.app/pelican/article_search?text=name"
    method="get">
            <label for="site-search">Search the site:</label>
            <input type="search" id="site-search" name="text"
                    aria-label="Search through site content">
            <button class="rounded-full w-16 hover:bg-blue-300">Search</button>
    </form>
    </section>

Putting it all together with datasette and Vercel

Here's where the magic starts. Publishing data to Vercel with datasette is extremely easy with the datasette plugin datasette-publish-vercel.

You do need to have the Vercel cli installed, but once you do, the steps for publishing your SQLite database is really well explained in the datasette-publish-vercel documentation.

One final step to do was to add a MAKE command so I could just type a quick command which would create my content, generate the SQLite database AND publish the SQLite database to Vercel. I added the below to my Makefile:

vercel:
    { \
    echo "Generate content and database"; \
    make html; \
    echo "Content generation complete"; \
    echo "Publish data to vercel"; \
    datasette publish vercel pelican.db --project=search-ryancheley --metadata metadata.json; \
    echo "Publishing complete"; \
    }

The line

datasette publish vercel pelican.db --project=search-ryancheley --metadata metadata.json; \

has an extra flag passed to it (--metadata) which allows me to use metadata.json to create a saved query which I call article_search. The contents of that saved query are:

select summary as 'Summary', url as 'URL', published_date as 'Published Data' from content where content like '%' || :text || '%' order by published_date

This is what allows the action in the form above to have a URL to link to in datasette and return data!

With just a few tweaks I'm able to include a search tool, powered by datasette for my pelican blog. Needless to say, I'm pretty pumped.

Next Steps

There are still a few things to do:

separate search form html file (for my site)
formatting datasette to match site (for my vercel powered instance of datasette)
update the README for pelican-to-sqlite package to better explain how to fully implement
Get pelican-to-sqlite added to the pelican-plugins page

The Well Maintained Test

2021-11-22T19:57:00-08:00

At the beginning of November Adam Johnson tweeted

I’ve come up with a test that we can use to decide whether a new package we’re considering depending on is well-maintained.

and linked to an article he wrote.

He came up (with the help of Twitter) twelve questions to ask of any library that you're looking at:

Is it described as “production ready”?
Is there sufficient documentation?
Is there a changelog?
Is someone responding to bug reports?
Are there sufficient tests?
Are the tests running with the latest <Language> version?
Are the tests running with the latest <Integration> version?
Is there a Continuous Integration (CI) configuration?
Is the CI passing?
Does it seem relatively well used?
Has there been a commit in the last year?
Has there been a release in the last year?

I thought it would be interesting to turn that checklist into a Click App using Simon Willison's Click App Cookiecutter.

I set out in earnest to do just that on November 8th.

What started out as just a simple Click app, quickly turned in a pretty robust CLI using Will McGugan's Rich library.

I started by using the GitHub API to try and answer the questions, but quickly found that it couldn't answer them all. Then I cam across the PyPI API which helped to answer almost all of them programmatically.

There's still a bit of work to do to get it where I want it to, but it's pretty sweet that I can now run a simple command and review the output to see if the package is well maintained.

You can even try it on the package I wrote!

the-well-maintained-test https://github.com/ryancheley/the-well-maintained-test

Which will return (as of this writing) the output below:

1. Is it described as 'production ready'?
        The project is set to Development Status Beta
2. Is there sufficient documentation?
        Documentation can be found at
https://github.com/ryancheley/the-well-maintained-test/blob/main/README.md
3. Is there a changelog?
        Yes
4. Is someone responding to bug reports?
        The maintainer took 0 days to respond to the bug report
        It has been 2 days since a comment was made on the bug.
5. Are there sufficient tests? [y/n]: y
        Yes
6. Are the tests running with the latest Language version?
        The project supports the following programming languages
                - Python 3.7
                - Python 3.8
                - Python 3.9
                - Python 3.10

7. Are the tests running with the latest Integration version?
        This project has no associated frameworks
8. Is there a Continuous Integration (CI) configuration?
        There are 2 workflows
         - Publish Python Package
         - Test

9. Is the CI passing?
        Yes
10.  Does it seem relatively well used?
        The project has the following statistics:
        - Watchers: 0
        - Forks: 0
        - Open Issues: 1
        - Subscribers: 1
11.  Has there been a commit in the last year?
        Yes. The last commit was on 11-20-2021 which was 2 days ago
12. Has there been a release in the last year?
        Yes. The last commit was on 11-20-2021 which was 2 days ago

There is still one question that I haven't been able to answer programmatically with an API and that is:

Are there sufficient tests?

When that question comes up, you're prompted in the terminal to answer either y/n.

But, it does leave room for a fix by someone else!

Styling Clean Up with Bash

2021-10-26T19:23:00-07:00

I have a side project I've been working on for a while now. One thing that happened overtime is that the styling of the site grew organically. I'm not a designer, and I didn't have a master set of templates or design principals guiding the development. I kind of hacked it together and made it look "nice enough"

That was until I really starting going from one page to another and realized that there styling of various pages wasn't just a little off ... but A LOT off.

As an aside, I'm using tailwind as my CSS Framework

I wanted to make some changes to the styling and realized I had two choices:

Manually go through each html template (the project is a Django project) and catalog the styles used for each element

Try and write a bash command to do it for me

Well, before we jump into either choice, let's see how many templates there are to review!

As I said above, this is a Django project. I keep all of my templates in a single templates directory with each app having it's own sub directory.

I was able to use this one line to count the number of html files in the templates directory (and all of the sub directories as well)

ls -R templates | grep html | wc -l

There are 3 parts to this:

ls -R templates will list out all of the files recursively list subdirectories encountered in the templates directory
grep html will make sure to only return those files with html
wc -l uses the word, line, character, and byte count to return the number of lines return from the previous command

In each case one command is piped to the next.

This resulted in 41 html files.

OK, I'm not going to want to manually review 41 files. Looks like we'll be going with option 2, "Try and write a bash command to do it for me"

In the end the bash script is actually relatively straight forward. We're just using grep two times. But it's the options on grep that change (as well as the regex used) that are what make the magic happen

The first thing I want to do is find all of the lines that have the string class= in them. Since there are html templates, that's a pretty sure fire way to find all of the places where the styles I am interested in are being applied

I use a package called djhtml to lint my templates, but just in case something got missed, I want to ignore case when doing my regex, i.e, class= should be found, but so should cLass= or Class=. In order to get that I need to have the i flag enabled.

Since the html files may be in the base directory templates or one of the subdirectories, I need to recursively search, so I include the r flag as well

This gets us

grep -ri "class=" templates/*

That command will output a whole lines like this:

templates/tasks/steps_lists.html:    <table class="table-fixed w-full border text-center">
templates/tasks/steps_lists.html:                <th class="w-1/2 flex justify-left-2 p-2">Task</th>
templates/tasks/steps_lists.html:                <th class="w-1/4 justify-center p-2">Edit</th>
templates/tasks/steps_lists.html:                <th class="w-1/4 justify-center p-2">Delete</th>
templates/tasks/steps_lists.html:                    <td class="flex justify-left-2 p-2">
templates/tasks/steps_lists.html:                    <td class="p-2 text-center">
templates/tasks/steps_lists.html:                        <a class="block hover:text-gray-600"
templates/tasks/steps_lists.html:                            <i class="fas fa-edit"></i>
templates/tasks/steps_lists.html:                    <td class="p-2 text-center">
templates/tasks/steps_lists.html:                        <a class="block hover:text-gray-600"
templates/tasks/steps_lists.html:                            <i class="fas fa-trash-alt"></i>
templates/tasks/step_form.html:        <section class="bg-gray-400 text-center py-2">
templates/tasks/step_form.html:            <button type="submit" class="bg-blue-500 hover:bg-blue-700 text-white font-bold py-2 px-4 rounded">{{view.action|default:"Add"}} </button>

Great! We have the data we need, now we just want to clean it up.

Again, we'll use grep only this time we want to look for an honest to goodness regular expression. We're trying to identify everything in between the first open angle brackey (<) and the first closed angle bracket (>)

A bit of googling, searching stack overflow, and playing with the great site regex101.com gets you this

<[^\/].*?>

OK, we have the regular expression we need, but what options do we need to use in grep? In this case we actually have two options:

Use egrep (which allows for extended regular expressions)
Use grep -E to make grep behave like egrep

I chose to go with option 2, use grep -E. Next, we want to return ONLY the part of the line that matches the regex. For that, we can use the option o. Putting it all together we get

grep -Eo "<[^\/].*?>"

Now, we can pipe the results from our first command into our second command and we get this:

grep -ri "class=" templates/* | grep -Eo "<[^\/].*?>"

This will output to standard out, but next I really want to use a tool for aggregation and comparison. It was at this point that I decided the best next tool to use would be Excel. So I sent the output to a text file and then opened that text file in Excel to do the final review. To output the above to a text file called tailwind.txt we

grep -ri "class=" templates/* | grep -Eo "<[^\/].*?>" > tailwind.txt

With these results I was able to find several styling inconsistencies and then fix them up. In all it took me a few nights of working out the bash commands and then a few more nights to get the styling consistent. In the process I learned so much about grep and egrep. It was a good exercise to have gone through.

djhtml and justfile

2021-08-22T00:00:00-07:00

I had read about a project called djhtml and wanted to use it on one of my projects. The documentation is really good for adding it to precommit-ci, but I wasn't sure what I needed to do to just run it on the command line.

It took a bit of googling, but I was finally able to get the right incantation of commands to be able to get it to run on my templates:

djhtml -i $(find templates -name '*.html' -print)

But of course because I have the memory of a goldfish and this is more than 3 commands to try to remember to string together, instead of telling myself I would remember it, I simply added it to a just file and now have this recipe:

# applies djhtml linting to templates
djhtml:
    djhtml -i $(find templates -name '*.html' -print)

This means that I can now run just djhtml and I can apply djhtml's linting to my templates.

Pretty darn cool if you ask me. But then I got to thinking, I can make this a bit more general for 'linting' type activities. I include all of these in my precommit-ci, but I figured, what the heck, might as well have a just recipe for all of them!

So I refactored the recipe to be this:

# applies linting to project (black, djhtml, flake8)
lint:
    djhtml -i $(find templates -name '*.html' -print)
    black .
    flake8 .

And now I can run all of these linting style libraries with a single command just lint

Prototyping with Datasette

2021-08-09T18:26:00-07:00

At my job I work with some really talented Web Developers that are saddled with a pretty creaky legacy system.

We're getting ready to start on a new(ish) project where we'll be taking an old project built on this creaky legacy system (VB.net) and re-implementing it on a C# backend and an Angular front end. We'll be working on a lot of new features and integrations so it's worth rebuilding it versus shoehorning the new requirements into the legacy system.

The details of the project aren't really important. What is important is that as I was reviewing the requirements with the Web Developer Supervisor he said something to the effect of, "We can create a proof of concept and just hard code the data in a json file to fake th backend."

The issue is ... we already have the data that we'll need in a MS SQL database (it's what is running the legacy version) it's just a matter of getting it into the right json "shape".

Creating a 'fake' json object that kind of/maybe mimics the real data is something we've done before, and it ALWAYS seems to bite us in the butt. We don't account for proper pagination, or the real lengths of data in the fields or NULL values or whatever shenanigans happen to befall real world data!

This got me thinking about Simon Willison's project Datasette and using it to prototype the API end points we would need.

I had been trying to figure out how to use the db-to-sqlite to extract data from a MS SQL database into a SQLite database and was successful (see my PR to db-to-sqlite here)

With this idea in hand, I reviewed it with the Supervisor and then scheduled a call with the web developers to review datasette.

During this meeting, I wanted to review:

The motivation behind why we would want to use it
How we could leverage it to do Rapid Prototyping
Give a quick demo data from the stored procedure that did the current data return for the legacy project.

In all it took less than 10 minutes to go from nothing to a local instance of datasette running with a prototype JSON API for the web developers to see.

I'm hoping to see the Web team use this concept more going forward as I can see huge benefits for Rapid Prototyping of ideas, especially if you already have the data housed in a database. But even if you don't, datasette has tons of tools to get the data from a variety of sources into a SQLite database to use and then you can do the rapid prototyping!

Contributing to Tryceratops

2021-08-07T00:00:00-07:00

I read about a project called Tryceratops on Twitter when it was tweeted about by Jeff Triplet

I checked it out and it seemed interesting. I decided to use it on my simplest Django project just to give it a test drive running this command:

tryceratops .

and got this result:

Done processing! 🦖✨
Processed 16 files
Found 0 violations
Failed to process 1 files
Skipped 2340 files

This is nice, but what is the file that failed to process?

This left me with two options:

Complain that this awesome tool created by someone didn't do the thing I thought it needed to do

Submit an issue to the project and offer to help.

I went with option 2 😀

My initial commit was made in a pretty naive way. It did the job, but not in the best way for maintainability. I had a really great exchange with the maintainer Guilherme Latrova about the change that was made and he helped to direct me in a different direction.

The biggest thing I learned while working on this project (for Python at least) was the logging library. Specifically I learned how to add:

a formatter
a handler
a logger

For my change, I added a simple format with a verbose handler in a custom logger. It looked something like this:

The formatter:

"simple": {
    "format": "%(message)s",
},

The handler:

"verbose_output": {
    "class": "logging.StreamHandler",
    "level": "DEBUG",
    "formatter": "simple",
    "stream": "ext://sys.stdout",
},

The logger:

"loggers": {
    "tryceratops": {
        "level": "INFO",
        "handlers": [
            "verbose_output",
        ],
    },
},

This allows the verbose flag to output the message to Standard Out and give and INFO level of detail.

Because of what I learned, I've started using the logging library on some of my work projects where I had tried to roll my own logging tool. I should have known there was a logging tool in the Standard Library BEFORE I tried to roll me own 🤦🏻‍♂️

The other thing I (kind of) learned how to do was to squash my commits. I had never had a need (or desire?) to squash commits before, but the commit message is what Guilherme uses to generate the change log. So, with his guidance and help I tried my best to squash those commits. Although in the end he had to do it (still not entiredly sure what I did wrong) I was exposed to the idea of squashing commits and why they might be done. A win-win!

The best part about this entire experience was getting to work with Guilherme Latrova. He was super helpful and patient and had great advice without telling me what to do. The more I work within the Python ecosystem the more I'm just blown away by just how friendly and helpful everyone is and it's what make me want to do these kinds of projects.

If you haven't had a chance to work on an open source project, I highly recommend it. It's a great chance to learn and to meet new people.

Contributing to django-sql-dashboard

2021-07-09T00:00:00-07:00

Last Saturday (July 3rd) while on vacation, I dubbed it “Security update Saturday”. I took the opportunity to review all of the GitHub bot alerts about out of date packages, and make the updates I needed to.

This included updated django-sql-dashboard to version 1.0 … which I was really excited about doing. It included two things I was eager to see:

Implemented a new column cog menu, with options for sorting, counting distinct items and counting by values. #57
Admin change list view now only shows dashboards the user has permission to edit. Thanks, Atul Varma. #130

I made the updates on my site StadiaTracker.com using my normal workflow:

Make the change locally on my MacBook Pro
Run the tests
Push to UAT
Push to PROD

The next day, on July 4th, I got the following error message via my error logging:

Internal Server Error: /dashboard/games-seen-in-person/

ProgrammingError at /dashboard/games-seen-in-person/
could not find array type for data type information_schema.sql_identifier

So I copied the url /dashboard/games-seen-in-person/ to see if I could replicate the issue as an authenticated user and sure enough, I got a 500 Server error.

Troubleshooting process

The first thing I did was to fire up the local version and check the url there. Oddly enough, it worked without issue.

OK … well that’s odd. What are the differences between the local version and the uat / prod version?

The local version is running on macOS 10.15.7 while the uat / prod versions are running Ubuntu 18.04. That could be one source of the issue.

The local version is running Postgres 13.2 while the uat / prod versions are running Postgres 10.17

OK, two differences. Since the error is could not find array type for data type information_schema.sql_identifier I’m going to start with taking a look at the differences on the Postgres versions.

First, I looked at the Change Log to see what changed between version 0.16 and version 1.0. Nothing jumped out at me, so I looked at the diff between several files between the two versions looking specifically for information_schema.sql_identifier which didn’t bring up anything.

Next I checked for either information_schema or sql_identifier and found a chance in the views.py file. On line 151 (version 0.16) this change was made:

string_agg(column_name, ', ' order by ordinal_position) as columns

to this:

array_to_json(array_agg(column_name order by ordinal_position)) as columns

Next, I extracted the entire SQL statement from the views.py file to run in Postgres on the UAT server

            with visible_tables as (
              select table_name
                from information_schema.tables
                where table_schema = 'public'
                order by table_name
            ),
            reserved_keywords as (
              select word
                from pg_get_keywords()
                where catcode = 'R'
            )
            select
              information_schema.columns.table_name,
              array_to_json(array_agg(column_name order by ordinal_position)) as columns
            from
              information_schema.columns
            join
              visible_tables on
              information_schema.columns.table_name = visible_tables.table_name
            where
              information_schema.columns.table_schema = 'public'
            group by
              information_schema.columns.table_name
            order by
              information_schema.columns.table_name

Running this generated the same error I was seeing from the logs!

Next, I picked apart the various select statements, testing each one to see what failed, and ended on this one:

select information_schema.columns.table_name,
array_to_json(array_agg(column_name order by ordinal_position)) as columns
from information_schema.columns

Which generated the same error message. Great!

In order to determine how to proceed next I googled sql_identifier to see what it was. Turns out it’s a field type in Postgres! (I’ve been working in MSSQL for more than 10 years and as far as I know, this isn’t a field type over there, so I learned something)

Further, there were changes made to that field type in Postgres 12!

OK, since there were changes made to that afield type in Postgres 12, I’ll probably need to cast the field to another field type that won’t fail.

That led me to try this:

select information_schema.columns.table_name,
array_to_json(array_agg(cast(column_name as text) order by ordinal_position)) as columns
from information_schema.columns

Which returned a value without error!

Submitting the updated code

With the solution in hand, I read the Contribution Guide and submitting my patch. And the most awesome part? Within less than an hour Simon Willison (the project’s maintainer) had replied back and merged by code!

And then, the icing on the cake was getting a shout out in a post that Simon wrote up about the update that I submitted!

Holy smokes that was sooo cool.

I love solving problems, and I love writing code, so this kind of stuff just really makes my day.

Now, I’ve contributed to an open source project (that makes 3 now!) and the issue with the /dashboard/ has been fixed.

All

Publishing content to Pelican site

2021-07-07T00:00:00-07:00

There are a lot of different ways to get the content for your Pelican site onto the internet. The Docs show an example using rsync.

For automation they talk about the use of either Invoke or Make (although you could also use Just instead of Make which is my preferred command runner.)

I didn't go with any of these options, instead opting to use GitHub Actions instead.

I have two GitHub Actions that will publish updated content. One action publishes to a UAT version of the site, and the other to the Production version of the site.

Why two actions you might ask?

Right now it's so that I can work through making my own theme and deploying it without disrupting the content on my production site. Also, it's a workflow that I'm pretty used to:

Local Development
Push to Development Branch on GitHub
Pull Request into Main on GitHub

It kind of complicates things right now, but I feel waaay more comfortable with having a UAT version of my site that I can just undo if I need to.

Below is the code for the Prod Deployment

name: Pelican Publish

on:
push:
    branches:
    - main

jobs:
deploy:
    runs-on: ubuntu-18.04
    steps:
    - name: deploy code
        uses: appleboy/ssh-action@v0.1.2
        with:
        host: ${{ secrets.SSH_HOST }}
        key: ${{ secrets.SSH_KEY }}
        username: ${{ secrets.SSH_USERNAME }}

        script: |
            rm -rf ryancheley.com
            git clone git@github.com:ryancheley/ryancheley.com.git

            source /home/ryancheley/venv/bin/activate

            cp -r ryancheley.com/* /home/ryancheley/

            cd /home/ryancheley

            pip install -r requirements.txt

            pelican content -s publishconf.py

Let's break it down a bit

Lines 3 - 6 are just indicating when the actually perform the actions in the lines below.

In line 13 I invoke the appleboy/ssh-action@v0.1.2 which allows me to ssh into my server and then run some command line functions.

On line 20 I remove the folder where the code was previously cloned from, and in line 21 I run the git clone command to download the code

Line 23 I activate my virtual environment

Line 25 I copy the code from the cloned repo into the directory of my site

Line 27 I change directory into the source for the site

Line 29 I make any updates to requirements with pip install

Finally, in line 31 I run the command to publish the content (which takes my .md files and turns them into HTML files to be seen on the internet)

Setting up the Server to host my Pelican Site

2021-07-05T00:00:00-07:00

Creating the user on the server

Each site on my server has it's own user. This is a security consideration, more than anything else. For this site, I used the steps from some of my scripts for setting up a Django site. In particular, I ran the following code from the shell on the server:

adduser --disabled-password --gecos "" ryancheley

adduser ryancheley www-data

The first command above creates the user with no password so that they can't actually log in. It also creates the home directory /home/ryancheley. This is where the site will be server from.

The second commands adds the user to the www-data group. I don't think that's strictly necessary here, but in order to keep this user consistent with the other web site users, I ran it to add it to the group.

Creating the nginx config file

For the most part I cribbed the nginx config files from this blog post.

There were some changes that were required though. As I indicated in part 1, I had several requirements I was trying to fulfill, most notably not breaking historic links.

Here is the config file for my UAT site (the only difference between this and the prod site is the server name on line 3):

server {

    server_name uat.ryancheley.com;
    root /home/ryancheley/output;

    location / {
        # Serve a .gz version if it exists
        gzip_static on;
        error_page 404 /404.html;
        rewrite ^/index.php/(.*) /$1  permanent;
    }

    location = /favicon.ico {
        # This never changes, so don't let it expire
        expires max;
    }


    location ^~ /theme {
        # This content should very rarely, if ever, change
        expires 1y;
    }

    listen [::]:443 ssl ipv6only=on; # managed by Certbot
    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/uat.ryancheley.com/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/uat.ryancheley.com/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

}

server {
    if ($host = uat.ryancheley.com) {
        return 301 https://$host$request_uri;
    } # managed by Certbot



    listen [::]:80;
    listen 80;

    server_name uat.ryancheley.com;
    return 404; # managed by Certbot


}

The most interesting part of the code above is the location block from lines 6 - 11.

    location / {
        # Serve a .gz version if it exists
        gzip_static on;
        error_page 404 /404.html;
        rewrite ^/index.php/(.*) /$1  permanent;
    }

Custom 404 Page

    error_page 404 /404.html;

This line is what allows me to have a custom 404 error page. If a page is not found nginx will serve up the html page 404.html which is generated by a markdown file in my pages directory and looks like this:

    Title: Not Found
    Status: hidden
    Save_as: 404.html

    The requested item could not be located.

I got this implementation idea from the Pelican docs.

Rewrite rule for index.php in the URL

    rewrite ^/index.php/(.*) /$1  permanent;

The rewrite line fixes the index.php challenge I mentioned in the previous post

It took me a really long time to figure this out because the initial config file had a location block that looked like this:

    location = / {
        # Instead of handling the index, just
        # rewrite / to /index.html
        rewrite ^ /index.html;
    }

I didn't recognize the location = / { on line 1 as being different than the location block above starting at line 6. So I added

    rewrite ^/index.php/(.*) /$1  permanent;

to that block and it NEVER worked because it never could.

The = in the location block indicates a literal exact match, which the regular expression couldn't do because it's trying to be dynamic, but the = indicates static 🤦🏻‍♂️

OK, we've got a user, and we've got a configuration file, now all we need is a way to get the files to the server.

I'll go over that in the next post.

Migrating to Pelican from Wordpress

2021-07-02T00:00:00-07:00

A little back story

In October of 2017 I wrote about how I migrated from SquareSpace to Wordpress. After almost 4 years I’ve decided to migrate again, this time to Pelican. I did a bit of work with Pelican during my 100 Days of Web Code back in 2019.

A good question to ask is, “why migrate to a new platform” The answer, is that while writing my post Debugging Setting up a Django Project I had to go back and make a change. It was the first time I’d ever had to use the WordPress Admin to write anything ... and it was awful.

My writing and posting workflow involves Ulysses where I write everything in MarkDown. Having to use the WYSIWIG interface and the ‘blocks’ in WordPress just broke my brain. That meant what should have been a slight tweak ended up taking me like 45 minutes.

I decided to give Pelican a shot in a local environment to see how it worked. And it turned out to work very well for my brain and my writing style.

Setting it up

I set up a local instance of Pelican using the Quick Start guide in the docs.

Pelican has a CLI utility that converts the xml into Markdown files. This allowed me to export my Wordpress blog content to it’s XML output and save it in the Pelican directory I created.

I then ran the command:

pelican-import --wp-attach -o ./content ./wordpress.xml

This created about 140 .md files

Next, I ran a few Pelican commands to generate the output:

pelican content

and then the local web server:

pelican --listen

I reviewed the page and realized there was a bit of clean up that needed to be done. I had categories of Blog posts that only had 1 article, and were really just a different category that needed to be tagged appropriately. So, I made some updates to the categorization and tagging of the posts.

I also had some broken links I wanted to clean up so I took the opportunity to check the links on all of the pages and make fixes where needed. I used the library LinkChecker which made the process super easy. It is a CLI that generates HTML that you can then review. Pretty neat.

Deploying to a test server

The first thing to do was to update my DNS for a new subdomain to point to my UAT server. I use Hover and so it was pretty easy to add the new entry.

I set uat.ryancheley.com to the IP Address 178.128.188.134

Next, in order to have UAT serve requests for my new site I need to have a configuration file for Nginx. This post gave me what I needed as a starting point for the config file. Specifically it gave me the location blocks I needed:

    location = / {
        # Instead of handling the index, just
        # rewrite / to /index.html
        rewrite ^ /index.html;
    }

    location / {
        # Serve a .gz version if it exists
        gzip_static on;
        # Try to serve the clean url version first
        try_files $uri.htm $uri.html $uri =404;
    }

With that in hand I deployed my pelican site to the server

The first thing I noticed was that the URLs still had index.php in them. This is a hold over from how my WordPress URL schemes were set up initially that I never got around to fixing but it’s always something that’s bothered me.

My blog may not be something that is linked to a ton (or at all?), but I didn’t want to break any links if I didn’t have to, so I decided to investigate Nginx rewrite rules.

I spent a bit of time trying to get my url to from this:

https://www.ryancheley.com/index.php/2017/10/01/migrating-from-square-space-to-word-press/

to this:

https://www.ryancheley.com/migrating-from-square-space-to-word-press/

using rewrite rules.

I gave up after several hours of trying different things. This did lead me to some awesome settings for Pelican that would allow me to retain the legacy Wordpress linking structure, so I updated the settings file to include this line:

ARTICLE_URL = 'index.php/{date:%Y}/{date:%m}/{date:%d}/{slug}/'
ARTICLE_SAVE_AS = 'index.php/{date:%Y}/{date:%m}/{date:%d}/{slug}/index.html'

OK. I still have the index.php issue, but at least my links won’t break.

404 Not Found

I starting testing the links on the site just kind of clicking here and there and discovered a couple of things:

The menu links didn’t always work
The 404 page wasn’t styled like I wanted it to me styled

The pelican documentation has an example for creating your own 404 pages which also includes what to update the Nginx config file location block.

And this is what lead me to discover what I had been doing wrong for the rewrites earlier!

There are two location blocks in the example code I took, but I didn’t see how they were different.

The first location block is:

    location = / {
        # Instead of handling the index, just
        # rewrite / to /index.html
        rewrite ^ /index.html;
    }

Per the Nginx documentation the =

If an equal sign is used, this block will be considered a match if the request URI exactly matches the location given.

BUT since I was trying to use a regular expression, it wasn’t matching exactly and so it wasn’t ‘working’

The second location block was not an exact match (notice there is no = in the first line:

location / {
        # Serve a .gz version if it exists
        gzip_static on;
        # Try to serve the clean url version first
        try_files $uri.htm $uri.html $uri =404;
    }

When I added the error page setting for Pelican I also added the URL rewrite rules to remove the index.php and suddenly my dream of having the redirect rules worked!

Additionally, I didn’t need the first location block at all. The final location block looks like this:

    location / {
        # Serve a .gz version if it exists
        gzip_static on;
        # Try to serve the clean url version first
        # try_files $uri.htm $uri.html $uri =404;
        error_page 404 /404.html;
        rewrite ^/index.php/(.*) /$1  permanent;
    }

I was also able to update my Pelican settings to this:

ARTICLE_URL = '{date:%Y}/{date:%m}/{date:%d}/{slug}/'
ARTICLE_SAVE_AS = '{date:%Y}/{date:%m}/{date:%d}/{slug}/index.html'

Victory!

What I hope to gain from moving

In my post outlining the move from SquareSpace to Wordpress I said,

As I wrote earlier my main reason for leaving Square Space was the difficulty I had getting content in. So, now that I’m on a WordPress site, what am I hoping to gain from it?

Easier to post my writing

See Item 1

Writing is already really hard for me. I struggle with it and making it difficult to get my stuff out into the world makes it that much harder. My hope is that not only will I write more, but that my writing will get better because I’m writing more.

So, what am I hoping to gain from this move:

Just as easy to write my posts
Easier to edit my posts

Writing is still hard for me (nearly 4 years later) and while moving to a new shiny tool won’t make the thinking about writing any easier, maybe it will make the process of writing a little more fun and that may lead to more words!

Addendum

There are already a lot of words here and I have more to say on this. I plan on writing a couple of more posts about the migration:

Setting up the server to host Pelican
The writing workflow used

Debugging Setting up a Django Project

2021-06-13T11:00:00-07:00

Normally when I start a new Django project I’ll use the PyCharm setup wizard, but recently I wanted to try out VS Code for a Django project and was super stumped when I would get a message like this:

ERROR:root:code for hash md5 was not found.
Traceback (most recent call last):
  File "/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py", line 147, in <module>
    globals()[__func_name] = __get_hash(__func_name)
  File "/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py", line 97, in __get_builtin_constructor
    raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type md5
ERROR:root:code for hash sha1 was not found.
Traceback (most recent call last):
  File "/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py", line 147, in <module>
    globals()[__func_name] = __get_hash(__func_name)
  File "/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py", line 97, in __get_builtin_constructor
    raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type sha1
ERROR:root:code for hash sha224 was not found.
Traceback (most recent call last):
  File "/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py", line 147, in <module>
    globals()[__func_name] = __get_hash(__func_name)
  File "/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py", line 97, in __get_builtin_constructor
    raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type sha224
ERROR:root:code for hash sha256 was not found.
Traceback (most recent call last):
  File "/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py", line 147, in <module>
    globals()[__func_name] = __get_hash(__func_name)
  File "/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py", line 97, in __get_builtin_constructor
    raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type sha256
ERROR:root:code for hash sha384 was not found.
Traceback (most recent call last):
  File "/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py", line 147, in <module>
    globals()[__func_name] = __get_hash(__func_name)
  File "/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py", line 97, in __get_builtin_constructor
    raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type sha384
ERROR:root:code for hash sha512 was not found.
Traceback (most recent call last):
  File "/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py", line 147, in <module>
    globals()[__func_name] = __get_hash(__func_name)
  File "/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py", line 97, in __get_builtin_constructor
    raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type sha512

Here are the steps I was using to get started

From a directory I wanted to create the project I would set up my virtual environment

python3 -m venv venv

And then activate it

source venv/bin/activate

Next, I would install Django

pip install django

Next, using the startproject command per the docs I would

django-admin startproject my_great_project .

And get the error message above 🤦🏻‍♂️

The strangest part about the error message is that it references Python2.7 everywhere … which is odd because I’m in a Python3 virtual environment.

I did a pip list and got:

Package    Version
---------- -------
asgiref    3.3.4
Django     3.2.4
pip        21.1.2
pytz       2021.1
setuptools 49.2.1
sqlparse   0.4.1

OK … so everything is in my virtual environment. Let’s drop into the REPL and see what’s going on

Well, that looks to be OK.

Next, I checked the contents of my directory using tree -L 2

├── manage.py
├── my_great_project
│   ├── __init__.py
│   ├── settings.py
│   ├── urls.py
│   └── wsgi.py
└── venv
    ├── bin
    ├── include
    ├── lib
    └── pyvenv.cfg

Yep … that looks good too.

OK, let’s go look at the installed packages for Python 2.7 then. On macOS they’re installed at

/usr/local/lib/python2.7/site-packages

Looking in there and I see that Django is installed.

OK, let’s use pip to uninstall Django from Python2.7, except that pip gives essentially the same result as running the django-admin command.

OK, let’s just remove it manually. After a bit of googling I found this Stackoverflow answer on how to remove the offending package (which is what I assumed would be the answer, but better to check, right?)

After removing the Django install from Python 2.7 and running django-admin --version I get

So I googled that error message and found another answers on Stackoverflow which lead me to look at the manage.py file. When I cat the file I get:

# manage.py

#!/usr/bin/env python
import os
import sys

...

That first line SHOULD be finding the Python executable in my virtual environment, but it’s not.

Next I googled the error message django-admin code for hash sha384 was not found

Which lead to this Stackoverflow answer. I checked to see if Python2 was installed with brew using

brew leaves | grep python

which returned python@2

Based on the answer above, the solution was to uninstall the Python2 that was installed by brew. Now, although Python2 has retired, I was leery of uninstalling it on my system without first verifying that I could remove the brew version without impacting the system version which is needed by macOS.

Using brew info python@2 I determined where brew installed Python2 and compared it to where Python2 is installed by macOS and they are indeed different

Output of brew info python@2

...
/usr/local/Cellar/python@2/2.7.15_1 (7,515 files, 122.4MB) *
  Built from source on 2018-08-05 at 15:18:23
...

Output of which python

/usr/bin/python

OK, now we can remove the version of Python2 installed by brew

brew uninstall python@2

Now with all of that cleaned up, lets try again. From a clean project directory:

python3 -m venv venv
source venv/bin/activate
pip install django
django-admin --version

The last command returned

zsh: /usr/local/bin/django-admin: bad interpreter: /usr/local/opt/python@2/bin/python2.7: no such file or directory
3.2.4

OK, I can get the version number and it mostly works, but can I create a new project?

django-admin startproject my_great_project .

Which returns

zsh: /usr/local/bin/django-admin: bad interpreter: /usr/local/opt/python@2/bin/python2.7: no such file or directory

BUT, the project was installed

├── db.sqlite3
├── manage.py
├── my_great_project
│   ├── __init__.py
│   ├── __pycache__
│   ├── asgi.py
│   ├── settings.py
│   ├── urls.py
│   └── wsgi.py
└── venv
    ├── bin
    ├── include
    ├── lib
    └── pyvenv.cfg

And I was able to run it

python manage.py runserver

Success! I’ve still got that last bug to deal with, but that’s a story for a different day!

Short Note

My initial fix, and my initial draft for this article, was to use the old adage, turn it off and turn it back on. In this case, the implementation would be the deactivate and then re activate the virtual environment and that’s what I’d been doing.

As I was writing up this article I was hugely influenced by the work of Julie Evans and kept asking, “but why?”. She’s been writing a lot of awesome, amazing things, and has several zines for purchase that I would highly recommend.

She’s also generated a few debugging ‘games’ that are a lot of fun.

Anyway, thanks Julie for pushing me to figure out the why for this issue.

Post Script

I figured out the error message above and figured, well, I might as well update the post! I thought it had to do with zsh, but no, it was just more of the same.

The issue was that Django had been installed in the base Python2 (which I knew). All I had to do was to uninstall it with pip.

pip uninstall django

The trick was that pip wasn't working out for me ... it was generating errors. So I had to run the command

python -m pip uninstall django

I had to run this AFTER I put the Django folder back into /usr/local/lib/python2.7/site-packages (if you'll recall from above, I removed it from the folder)

After that clean up was done, everything worked out as expected! I just had to keep digging!

My First Python Package

2021-06-06T18:11:00-07:00

A few months ago I was inspired by Simon Willison and his project Datasette and it’s related ecosystem to write a Python Package for it.

I use toggl to track my time at work and I thought this would be a great opportunity use that data with Datasette and see if I couldn’t answer some interesting questions, or at the very least, do some neat data discovery.

The purpose of this package is to:

Create a SQLite database containing data from your toggl account

I followed the tutorial for committing a package to PyPi and did the first few pushes manually. Then, using a GitHub action from one of Simon’s Datasette projects, I was able to automate it when I make a release on GitHub!

Since the initial commit on March 7 (my birthday BTW) I’ve had 10 releases, with the most recent one coming yesterday which removed an issue with one of the tables reporting back an API key which, if published on the internet could be a bad thing ... so hooray for security enhancements!

Anyway, it was a fun project, and got me more interested in authoring Python packages. I’m hoping to do a few more related to Datasette (although I’m not sure what to write honestly!).

Be sure to check out the package on PyPi.org and the source code on GitHub.

RyanCheley.com

A Giant Pain in the Ass

Migrating to Hetzner with Coolify

What I did

Original State

Current State

Setting Up Hetzner

Coolify: setting up projects, environments, resources

Starting with Nixpacks

Outline of Migration steps

Switching to Dockerfile/docker-compose.yaml

Writing the Dockerfile and docker-compose.yaml files

Upgrading all the things

Migrating database servers

Starting with UAT

Backups

Coolify dashboard thing

What this allows me to do now

Was it worth it

What's next?

Year in Review 2025

Professional

Django and Python

Technology

Personal

Music

Hockey

Baseball

Family

Conclusion

Details on My Candidate Statement for the DSF

Getting an Executive Director (ED) to help run the day-to-day operations of the DSF

Identifying small to midsized companies for sponsorships

What an Executive Director Would Enable

My First 90 Days as Treasurer

Implementing a formal strategic planning process

Setting up a fiscal sponsorship program to allow support of initiatives like Django Commons

Who This Helps

Why This Matters

Moving Forward Together

Deploying n8n on Digital Ocean

Create and configure the VPS

Install Docker

Configure DNS

Create Docker Compose configuration

Add n8n to Docker Compose

Start the containers

Complete the setup

Why We Need to Stop Fighting About AI Tools and Start Teaching Them

Migrating to Raindrop.io

A New Project at Work

Updated TIL

Fun with MCPs

What is Model Context Protocol (MCP)?

(Potential) Real World Example

Enter an MCP with a Local LLM

Application of MCP with the Example

Firebirds 2024-25 Season

Uptime Kuma and Claude

The Invisible Decision-Makers: Why Systems Ignore Their Users

The Origin of Systems

The Illusion of Success

Questioning Purpose

User-Hostile Systems

Why Bad Systems Persist

Conclusion

Process, People, and Priorities

Defining Terms

Process

People

An Example

Priorities

Technical Solutions to People Problems

The situation

The Solution

The Outcome

How to Watch a Hockey Game - Reading the Standings

Game Outcomes

A Concrete Example

Winning Percent

Writing the `Dockerfile` and `docker-compose.yaml` files