Stephen Castro-Starkey

An interesting side effect of doing more ensemble programming — you find yourself waiting less. Also, the people that are not in the ensemble find themselves waiting more. Especially if they need someone who happens to be in the ensemble.

Today I learned while reading https://blog.janetacarr.com/software-design-patterns-in-clojure/ that you can name your lambdas in Clojure! 🤯

I will be doing this from now on!

Today I learned, with a bunch of digging, how to make clojure-lsp work on my computer. Since I have clojure installed in a "weird" place for linux, zed couldn't find it. And the editor's documentation doesn't talk about how to fix that anywhere. Figured I'd document my findings here for posterity.

Found a bug in my old webmention code (this post wasn't working!) for this blog and fixed because I don't know why -- it's not like this blog is gonna live much longer. As I was about to deploy the app I noticed it's been running nonstop for 5 months. Not bad!

I feel like I've pretty much gotten to a place where I can rebuild this blog using streamful. It will be an interesting challenge! The only issue I have right now is that my only mac computer is super slow. Also, ios development is hard for me right now. But I shall do my best!

Today I had no energy to add new functionality but I did to do some cleanup! Proud of myself as I uncovered some shortcuts I took that I'm not proud of. Had to back in test coverage! If I were really taking small steps and only writing enough code to make a test pass this would not have been necessary methinks. Yuck.

A bunch of commits and files showing how you can make threading happen in streamful. Likely these aggregators will end up being default for all servers. Tho they may evolve a bit, like, for example, only keeping the last X messages in a thread. The rest of them might end up being a search or something?

Still working on aggregates — slowly but surely! I have successfully compiled text with only a whitelisted set of allowed symbols, which is quite handy. Figuring out how to do the hot swap from pending to active and retroactively applying them is next. It’s almost like I’m building a database engine. Which I am not at all qualified for. What could possibly go wrong?

This commit is quite thorough. It breaks down all the data structures and how one would produce a message that the streamful server could understand.

Today's coding session I decided I needed to write more tests. Which naturally led to adding a configurable maximum upload size. Which is good!

I'd call this.. exploratory unit testing. :-)

re this earlier post, I was wrong. Soon as I published it I found the energy to build a new thing (API for listing available streams). It took a surprisingly short amount of time (~36 minutes from start to push)!

Did some cleanup work today but didn't build anything new into streamful. But I think that's ok. The energy will return!

Added encryption but not in the usual way. More like SSH than TLS. The eventual intention is to use a combination of DNSSEC and TXT records to prove that a server belongs to somebody. The server's public key is in the TXT record! There is prior art of a similar approach.

So when you establish a session, we use the libsodium key exchange protocol to set up a secure session. Then all future messages are encrypted between client and server safely!

Well.. I hope. I am not a security person. But they say if you use libsodium you probably can't go wrong?

Ha!

This commit was a huge leap forward in my understanding of how Netty works under the covers. Namely, we were decoding arbitrary messages in a long running connection where sometimes you get a complete message and sometimes you don't. This was the moment when, for the first time, the "sometimes you don't" actually happened.

Today all I accomplished was protecting the privacy of a stream at a very basic level. Server admins can still see everything though. Wasn’t too terribly difficult!

Today I started in on storing and retrieving messages in a stream. Makes me wonder: how do I protect against malicious server admins hijacking my stream? Was thinking of having every message posted to a stream also including a hash of the stream’s “root message.” Trying to figure out how this can be hijacked if all messages are signed and can be verified by other clients.

Took a bit of a break but this morning I sketched out the basic storage mechanism for stream configuration.

The beginnings of public/private key authentication are in the code!

Long-ish post about the next steps on streamful — the bottom two layers are explained in more detail. Lowest layer is fairly stabilized!

re my post about dependency management, the issue was Jitpack. Their repository is not responding. That's problematic!

I ended up swapping out the one library I was using in jitpack with one that was hosted in clojars.

It's been a while since I updated my blog software itself! I get a daily email telling me all of my periodic activities and when was the last time I did that thing. Sometimes things didn't show up at all. I think that's because I was sending the activity report at midnight, and basically all the activities also key off the midnight hour. So, I figured maybe if I send the report a few minutes after midnight, I'd get a more accurate picture of how long it's really been since I did each activity.

Very small change.

The fun part -- part of the build process for the app (the part that checks for outdated libraries) has basically stopped working. So I had to exclude it. Gonna have to debug that more!

Today I learned about “learning debt” in which code reviews cause negative feedback cycles discouraging people from taking the time to deeply learn a code base. This is due to biases around how people reading your code only seem to value the code that you wrote instead of the effort you put into learning what was necessary to make that code change effectively.

If only there was a method that flipped that narrative on its head. Some way to get code reviews while also encouraging and amplifying learning!

The fun part about tech debt: the cost cannot be quantified until it comes time to pay it. And the longer you wait, the more uncertain it becomes.

Don't write the devil a blank check.

Yesterday I discovered that SQLite isn’t all that reliable at maintaining integrity of FTS5 virtual tables. Following their documentation to create a content backed virtual table with the recommended triggers lead to database corruption after just 5 operations.

Surprising how little I’ve been reporting on my coding adventures. Coffee Time is making good progress! Have been working on little things after getting feedback from conversations. Also started thinking about wiki!

There is a ton of infrastructure I built around the old lean coffee app that relied on Selenium WebDriver. Now that I’m developing on Linux for the first time and struggling with it, I’m realizing I should probably throw all that away. Spent many hours trying to figure out why my tests wouldn’t run at all.

I feel like a return to simplicity is in order.

I’ve been thinking a lot about authentication in My Cafe. It’s starting to look like you will register/logon using your email address, and then afterwards you can add a passkey so you can login that way if you want to.

Almost done implementing email based authn!

Feeling a bit down lately. But maybe that's just sleep deprivation? Have made some small edits to my legacy lean coffee app in preparation for bringing a few friends down memory lane as we consider rebuilding it! Or maybe I'll find a way to hack up the old one to make it easier to self service? Not sure what I want to do really. But, I'm certain I don't like compiling JavaScript. Ha!

Visit the old app if you want!

My thoughts so far. Third party auth or local? Simplicity or complexity? Or eventually both?

The new cafe app is coming along nicely. Just copied over the site config code from My Life and tweaked it a bit to correct some sins I've discovered. Next up: authentication!

Earlier I wrote that I would try out SuperTokens. Sadly, I don't think I want people to have to manage multiple server processes, and they require a backend written in either golang, nodejs, or python. Our backend is Java-based. Guess we'll have to try something else!

Well, as I said yesterday, I tried to integrate Ripley. It didn't work. Very much needs to plug into internals I don't want to work with (like http-kit).

So now I'm looking at Inertia, Unpoly, Twinspark, and htmx all thanks to this blog post

🤷‍♂️

Small update to the blog -- I no longer show my daily activities on the front page. You'll need to click through to one of the categories to see all that stuff.

I was kind of proud of how easy it was to add this feature -- it's a mix of code and template magic, so I was able to deploy the code without any behavior. Then I updated the home page template and it showed up!

I seem to have reached the limit of my energy to improve my blog. Perhaps that's for the best! Now I'm gonna focus a bit on implementing a small Lean Coffee web app. Or mobile app. Not sure exactly. But it will be entirely self hosted and open source and free!

So, after running it on Jetty, the server doesn't seem to need as much memory, but it's still a lot. After about an hour on aleph, the heap was about 340m in size. On Jetty after about an hour it was at 192m.

Hoping that this issue gets me some clarity!

And now we are back on jetty. Gonna run it a while in a constrained memory environment (192m heap) and see how it goes.

My blog is now running on aleph instead of jetty. Interestingly, it uses up a lot more memory. Capping the heap at 340m makes it performant enough. Was able to cap the jetty implementation at 192m.

Perhaps it is more resource efficient in other ways? You can see what I had to change.

I might try out yada next. But it seems to be a lot more opinionated.

My brain is currently noodling on why I’m using nginx to route traffic to fossil and so forth, while simultaneously allowing mylife to host its own SSL cert, thus losing out on tracking which IP is hitting the app at any given time. Looking for alternative approaches! I don’t want to go deeper into virtual networking but I may have to!

Thinking more about the social aspect -- what would it take to discover all the necessary information if someone else's blog doesn't support the webmention protocol?

Today's work on My Life revolved around making it easier to log in and stay logged in. I realized my phone was purging session cookies basically every day, so I changed the cookie management to set explicit expiration dates. And extend those expiration dates every time you use the app. Therefore, so long as you keep using the app you won't get logged out.

Made a bit more progress on "New Activity" -- had to refactor the front page to have a "new..." link instead of separate links for each "new" thing I could do. It just didn't look good on mobile! On the new options page I'll add "new activity" or just put the activities directly there. Not sure!

Almost ready to wire up the activity code to the UI! Soon I'll be able to configure and execute recurring activities.

The morning as a bit stressful with a tornado warning but all is well. Did some things today:

• ✔ Rowing
• ✔ Standing Meditation
• ✔ Wrote some code in My Life to set the foundation for tracking recurring activities (habits)

I can now export the entire blog, including page and post edit history. Very happy about that! I'll still keep my pages checked in so I can easily use a fancy editor to modify them, at least for now, since I'm still stuck on not having any JavaScript.

I now have a local Nexus repository running that allows folks to download the docker images I publish for My Life. However, the links are all wrong because I don't have it configured properly. It'll get there! I hope? :-) Will set up a link to it soon as I have it all figured out. Or shut it down because it isn't worth it. Likely this is a path I didn't need to tread down, as nobody really wants to run this software anyway except me!

Did a bit of research on what it takes to host the container images for My Life today. It wasn’t the easiest thing in the world! And that’s basically all I accomplished today. Did have some fun playing Baldur’s Gate 3 with my best buddy though! So, a bit of coding and a lot of rest.

Feels fine. :-)

Need to get back on track and find a smaller bite to take.

We have upgraded to JDK 21! I attempted to make this thing work with native image compilation using GraalVM, but that failed miserably. I didn't try too hard, so maybe if I did somehow find the energy to build that skillset it would happen later?

re: https://notes.jatan.space/2023/12/20/im-so-happy.html, I agree, but I wish it was easier to build small apps that you can self host that also federate. I'm doing an experiment using just IndieWeb constructs and avoiding ActivityPub because of how ill defined it is. Fingers crossed it goes somewhere!

Reporting on the status of my blog's infrastructure. How much CPU, disk space, and memory is it using?

I never thought I would find a way to test the job scheduler but when I broke it and didn’t notice for a day I realized I had better! So I did.

Wrote a unit test to validate all the schedules generated a next time to run successfully. That made me feel quite safe!

So I guess that five minute task actually took a bit longer. ;-)

How dare my family interrupt my coding with trips to beautiful places in Colombia?

re: cache invalidation, a simple cron job every 6 hours to clear all caches seems a decent stopgap. Got it done in 5 minutes!

Cache invalidation continues to be a bane. The charts don’t change when the day goes on. Ha! But I keep learning as I use my software. Kind of fun!

Finally added time zone support to the app so now my blog is displaying Eastern Time instead of GMT. You’re welcome. 😜

Now we are passing quite a few webmention tests! It's amazing what you can accomplish when you get a stomach bug on vacation.

There was one issue with webmention receipt that I found when testing in production. It had to do with using a function that relied on the internal domain/port setup, which was not the right thing to do. Only worked while testing locally. Fun times! Anyway, it should now be working!

It is surprisingly complex to receive webmentions, at least if I want to display some useful information about them on my posts. Have been working on it all day!

So far I can parse some simple microformats2 info and may fall back to open graph if I don’t find anything. Generating a unique ID requires my first use of crypto (for generating hmac hash of a webmention).

Just gotta store them and then build the white / black listing logic. And finally attaching the approved ones to the correct post and rendering them.

In which I admit that I chose not to use TDD and why. Trade offs!

Also, cache invalidation is the bane of my existence. There is something to be said about a blanket content cache. Much simpler than what I’m working with. Perhaps in the rewrite of tagging I will switch to that! Not sure.

We have tags!

Sort of.

Ok so I have written some code but not checked it in. This is what I suppose I would call a “spike solution” but with a fun twist that it’s running against a production database. No idea how that will break things if I roll the code back!

I don’t want to roll it all back. But it wasn’t test driven! So I gotta try test driving it so I don’t miss anything. It was a LOT of functionality.

Also I need it to keep me logged in longer on iOS. Not sure why the sessions don’t last as long.

Feels like the language and stack I picked make it want to be a multi tenant thing instead of many small things running independently.

That is if I want to minimize electricity waste! Will need to see how much memory it takes to run this thing and do some thinking.

Aaron has some adorable kitties! This makes me want to be able to upload images to my blog now so I can post some more photos of our babies.

By the way, I need to thank him for his patience as I fumble around with all this microformats stuff!

Sometimes I test in production and things go horribly wrong. Today was one of those days. Thank goodness I was using a bitemporal database! Today, I needed to rewind history on all my blog posts while testing sending of webmentions as I was accidentally overwriting the creation time of a post when setting a flag that I had successfully sent all its webmentions. Lesson learned!

Side benefit -- I now have the ability to show the history of a post and so forth. Sometimes taking off the safety net forces you to garner unexpected benefits. Of course, one must also have made good decisions leading up to the moment of removal of said safety net. 😅

Oooh look! I have a profile on wedotdd.com

Very nice.

A confession about how I broke this website by not using TDD properly!