amiantos.net

  • Early into 2023 I decided I would make my own blogging CMS (Content Management System) and started thinking about it enough that I came up with a name and even registered a domain name: Postalgic(.app). But then it seems like I got distracted, and it wasn’t until over two years later (about a month ago) that I sat down to build Postalgic for real.

    The Inspiration

    The main motivation for me to finally finish this app was a pretty negative experience I had with WordPress.com hosting for my resurrected music blog at staires.org. I’ve been using WordPress most of my life, so I figured I would support the company and pay for their managed hosting solution. For this blog (the one you’re reading), I use AWS Lightsail, which is basically just a tiny VPS and I’ve never been super happy with it as I had to rig it up to reboot every night or else my blog goes down, which is weird. So, it felt like it was a good opportunity to try something else out, and even though WordPress.com seemed absurdly expensive to me, I went for it.

    All was well was at first, but then one day I started running into an issue where, as I was writing a post, the entire paragraph I was writing would just disappear. I contacted their support about it (I was a paying customer after all) and during diagnostics, it seemed like it may be because of my VPN (which is hosted by AWS, and is frequently blocked by websites for appearing like bot traffic). But, even with the VPN off, it kept happening. Long story short, based on this forum thread of people having the same issue, it seems to be caused by a shortcut added to the editor, Shift+Backspace, which will delete the entire paragraph.

    They’ve allegedly reverted the change, but it took five months? Maybe more? And I don’t really believe that explanation, because I type in a very strange manner and I am pretty certain I have never accidentally hit Shift+Backspace at the same time. Additionally, this entire time the bug has never effected this WordPress instance, which is not hosted on WordPress.com. So, the whole thing feels fishy to me and left me with a very bad taste in my mouth, that I was paying so much money for a WordPress instance that couldn’t even use plugins and also getting a buggy and inferior version of the product.

    At the same time, I always felt like WordPress’ mobile experience was pretty garbage, the writing experience on mobile feels pretty terrible. WordPress also just isn’t really suitable as a replacement for something like Twitter or Mastodon, either, because of its strong focus on long-form content and CMS-like features. I don’t always want to write a long post, sometimes I just want to toss something off without thinking a lot about it, and doing that in WordPress just always has felt wrong to me.

    I started blogging back in 1999 with a service called, you guessed it, Blogger. If you described Blogger today, you’d call it a remotely hosted content management system and static site generator, tailored for journaling or blogging. This was before the idea of “tags” existed on the internet, so it was pretty barebones. You logged in, you picked your blog, and you wrote posts, and then it would generate the HTML for your site, connect to an FTP server of your choice, and upload your site to it. In some ways, it was ahead of its time, being a static site generator before anyone had ever thought of the term ‘static site’.

    Fast forward to 2023, there’s a lot of static site generators out there, but they’re all kind of a pain in the ass. I used Gatsby for this blog for several years, but writing posts into Markdown files, having to make decisions myself about post slugs, then hoping that I don’t end up in dependency hell trying to build the site for testing, and having to commit the files to a git repo, which makes updating on-the-fly pretty much impossible… it was not convenient, it was not fun.

    I wanted to be able to update my site whenever I wanted, wherever I was, and I didn’t want to deal with any annoyances whatsoever–no poorly designed apps, no being overcharged, no server maintenance, no dependencies, no workflow, no hacking concerns, no social media brouhahas over their egomaniac CEO…

    The Build

    About the same time I was deciding to work on Postalgic for real, I was turned onto Claude Code by a coworker. After using it for a couple small projects at work, I could feel the way I think about programming shifting a bit. This may be a contentious opinion in some circles (especially with some of the people over in the Hacker News comments), but the worst part of programming is often writing a bunch of boring ass code. There are some wackos out there who think the actual writing of code is the fun part, and I think those people imagine themselves to be doing some sort of ancient mental tai chi zen buddhism self-improvement “digital jazz” type bullshit, and I also think they are very wrong.

    To me, the fun parts is everything except the writing of the code. I enjoy having the idea. I enjoy envisioning and building out the idea into a full project. I enjoy the problem solving aspects of building the app: how do I design the data model, how do I architect the app, what underlying technologies to use, etc. I enjoy doing code review, the “theorycrafting” aspect of software design, where you consider whether every bit of code is doing its job and won’t bite you in the butt in the future. I enjoy doing QA and dogfooding the app, to see if it accomplishes the task its supposed to, and whether it does it cleanly and in a non-annoying way.

    What I do not particularly enjoy, but do not hate by any measure, is writing out long reams of same-y code that I’ve written out hundreds of times before; or googling answers to the same questions I’ve googled hundreds of times before. That code, it’s in my head, it’s getting it from my head and into the computer that is nothing but a chore. It’s busywork. If only I could build software and skip that annoying step… If only I could split myself in half and make that other sucker write all the code for me… If only…

    Well, spoiler warning, that’s what Claude Code is. Claude Code is the poor son of a bitch who has to write all my code for me. I give it my ideas, as best as I can (which is quite good, because I am nothing if not an effective communicator; look at you, reading this, how many words have I written so far, and you’re still reading? What’s wrong with you?) and then Claude Code semi-effectively writes the code for me, and it does so quite quickly. It can implement an entire new screen in an app for me, data model, functionality and all, while I brush my teeth. (Assuming I remembered to hit Shift+Tab before I walk away and I don’t come back to it asking for permission to make its first edits.)

    I don’t want to overstate it, Claude Code is not flawless. When writing SwiftUI specific code, it can be quite bad, as it’s full of muddled information from years of SwiftUI getting annual updates and cause huge changes to the most effective way to accomplish certain basic tasks. It’s not smarter than me, that’s for sure, and frequently makes mistakes quite like a poorly trained junior engineer could, especially if given poor instruction.

    I ran into several scenarios where, because I did not fully understand the problem scope, Claude Code didn’t seem to understand it either, and it spun its wheels (and burned money) building out nonfunctional features that it could not get into functioning state. I learned the hard way (by burning money, several times) that if Claude Code can’t fix its own code within 1 or 2 tries, it’s better for me to just throw everything it did out, blame myself, and fix whatever is wrong with my prompting. Usually this meant I needed to go do some of my usual feature building research myself, finding the most right solution and some implementation examples of it, which I then feed into Claude Code so that it knows everything that I know.

    Let’s cut the chase. Claude Code touched every single part of this project, and saved me a massive amount of time. Not only that, it gave me a lot of confidence that I did not previously have as an engineer. This is saying a lot, because I was a very confident engineer already, but my grasp did not reach as far as it could because I am inherently quite lazy. Because Claude Code takes so much of the busy work out of software engineering, it’s really taken some shackles off of my willingness to let ideas stretch and grow. All that’s left for me is the fun parts, so every idea seems fun to me now, and not full of drudgery and jockeying code.

    I really love Claude Code and I am now pretty fiercely devoted to it. I am very excited to see where the future leads for app development, and I say that without trying to be a soothsayer, which inevitably leads to forecasting doom and the destruction of the human race.

    The App

    Postalgic is a fully featured CMS and static site generator for iPhones. It’s basically as close to Blogger.com from the year 2000 as you can get, with some extra modern features, like full category and tag support, and rich ’embeds’ like YouTube, link share previews, and image galleries. It also supports several publishing methods: you can upload the site via SFTP, AWS S3, or push it to any git repo (for GitHub Pages support and more). It has a fully customizable templating system, and comes with a default theme which has a UX for customizing its colors.

    I’m already using it to host my resurrected journaling domain, my resurrected music blog domain, and the devlog for the app. A friend of mine is using it for a journal and a card game blog.

    The Postalgic.app website, which was built and written essentially entirely by Claude Code, can tell you more about the app, but I suppose the screenshots I came up with for the App Store could tell you even more.

    You can buy Postalgic on the App Store right now! Who wouldn’t want to?

    The Future

    This is really only the start of my plans for Postalgic. I intend for there to be two more phases of the project, though they’re somewhat conjoined. One is that I want to offer paid hosting under the .postalgic.app domain, so people can pay a very low fee to get a username on the domain and be able to upload their site there. The second is that I want to allow Postalgic blogs to submit their posts to what I’ve been calling “Postalgic Portal” in my head, which’ll be a non-algorithmic feed of posts people have submitted from their own blogs, meant to allow for discovery of other Postalgic blogs.

    I have some other things I want to do first, or should do first, like doing maintenance on my other apps that are on the App Store, but these things will hopefully be a fast follow, assuming I don’t get too hung up on making decisions about how and where to host the backend services needed for it. I have not built out a SaaS myself, I just help run one professionally, so it’ll be a slightly new experience for me.

    That’s it for now, see you next time.

  • I’m experiencing something of a very minor personal mental health crisis so far this year because it’s come to my attention that everything I like really sucks.

    It came into extra sharp focus this past week as I have been playing Avowed, the new RPG from semi-renowned game studio Obsidian. The game’s been met with generally positive reviews (81 on OpenCritic, and 81 on Metacritic, and 75-82 on Steam) that highlight the game’s exciting combat, gorgeous world design, inventive world design that encourages exploration, the interesting story, and high quality voice acting, while also noting its shortcomings in some areas when compared to certain classic RPG games.

    I’m about 30 hours into Avowed, and I can agree with that consensus, though I do not think the areas where it ‘falls short’ are at all important, so I’d rate the game a little bit higher than the critic average. It’s a solid… 88 out of 100? Not much of a difference. Regardless, the critical consensus and my experience of game so far jive completely. Avowed is a good game!

    However, there’s a very vocal contingent of gamers online that seems to believe Avowed is a very terrible game. I have seen people say that it is “absolute dogshit”, and there are numerous videos on YouTube dedicated to tearing the game apart for every perceived flaw, with no attention paid to any ways that the game may be otherwise good. One video I watched featured a narrator shouting almost the entire time, giving the impression that he was completely enraged by Avowed–very reminiscent of Alex Jones.

    In online communities where gamers congregate, it seems as though this opinion is held by the majority. It’s not possible to mention that you are enjoying Avowed and not have someone appear to tell you that the game is actually really bad. It is pervasive and unavoidable, and Avowed joins a list of other games that have been on the receiving end of treatment like this from the greater gaming community in the past couple years. If you go to any site that allows user reviews, you will see this pattern repeated: there are a lot of positive reviews from people who are enjoying the game; and an awful lot of negative reviews from people whose experience of the game is so much the opposite that it doesn’t quite make sense.

    And that’s where I start to feel my mind fracturing a little bit into pieces. I have played and am playing this game, so I know that it’s fun. There’s also a critical consensus that the game is fun. But then there are user reviews that say, without getting bogged down into too many details, that the exact systems and gameplay that other people enjoy, are actually total and complete garbage. How do you get these two perspectives to line up? Is the combat “fun and dynamic the whole game”, or is it “boring, stale, and one note from the very start”? How can both of these things possibly be true at the same time?

    They can’t be, and they’re not. And it’s because one side isn’t gaming in good faith. They are not interested in enjoying a video game for what it is, because their goal is not to promote video games they like and further the craft. Their goal is to disparage a game that they believe has rejected them in some way, and they’ve found a way to weaponize the language of game critique to attack games while trying (and usually failing) to mask what they’re really mad about.

    For example, numerous of these very negative takes end up mentioning that they’re upset that the characters are “ugly”, and some of the videos outright state that they’re upset that the female characters do not have ample cleavage and aren’t sexualized enough. They see modestly dressed female characters and they believe that there is a “woke agenda” that is motivating game studios to remove sexualized female characters from video games, which goes back to the Gamergate 1.0 days. But since then, a lot of gamers have learned that you just get called a “gooner” if you seem overly fixated on wanting female characters to be sexy, so they mask it by saying that the “character design” is bad.

    You’ll also come across reviews complaining explicitly about the game “forcing an agenda”, which is a reference to the fact that there are “woke” topics in the game, like having a gay character, or a side quest that allows you to fight against a government that has restricted access to birth control and abortion. Never mind that many games throughout history have had gay characters or featured storylines that would strike contemporary audiences as ‘woke.’ Fantasy settings have long been used to explore concepts like sexism and bigotry.

    The fact that Avowed, and other games, address these sorts of topics in their stories is received as an “agenda” being pushed on these gamers in yet another way. Not only has the “woke agenda” taken away the always-nearly-nude female fantasy characters from them, it’s also encouraging them to think and possibly care about homosexual, transgendered, and other minority groups when they’d rather live in a world where those kinds of people do not exist. If the negative review doesn’t outright say this explicitly, they mask it by saying that the “writing is bad” or the “dialogue is cringe”.

    Avowed is a good game, but there is a culture war going on in gaming right now that closely mirrors what is going on in politics around the world, and it’s all being driven by social media algorithms that purposefully feed people inflammatory content to drive engagement. There are people out there who are paying their bills by building out this sort of misinformation campaign and creating a culture of outrage and vindictiveness around video games. And, sadly, there is a very large audience out there that is susceptible to this kind of content and it has taken a hold over them. The result is a segment of gamers whose entire identity revolves around impassioned hatred of very specific games.

    …and holy fuck, it is depressing.

    It doesn’t seem like it’s very difficult for me to find or otherwise be exposed to content in which I learn that someone has a very strong dislike of something I personally quite enjoy. In fact, it’s starting to feel like I am immersed in a sea of people who really do not like the things I like, and that no place on the internet is safe from some jerk-wad popping their head up to tell me, “Hey, that thing you like, it fucking sucks and you are a nu-male soyboy beta cuck for liking it!”

    It’s not fun. It’s not exciting. It’s actually really disappointing. This behavior makes it difficult to experience a game in a vacuum, because the negative takes immediately put me on the back foot, and instead of being able to view the thing I enjoy solely through the lens of my own personal enjoyment, I begin to think of ways to defend the thing against the nonsensical and usually totally baseless attacks against it.

    In some cases, this feels impossible and puts a person into a position that feels strange. How am I supposed to defend a video game from someone who is upset that the female characters aren’t sexy enough? If I was telling someone about a video game I like, I would probably not even mention how attractive I find the characters, but now that there are people out there complaining about it, and I want to defend the game, so I feel like I have to speak up and go, Hey, actually, the characters in the game are pretty attractive! which then raises the question, “Why do you care if the characters are attractive, you 40-year-old creep?”

    I don’t! I’m not some fucking loser who is obsessed with how attractive video game characters are! I swear! I wouldn’t even mention it if it wasn’t for that fucking guy over there!

    It’s ridiculous that the dialogue around video games has come to this, that the people who love and enjoy video games end up feeling like we have to defend games against outlandish accusations and conspiracy theories. We are put in a position where we feel the need to defend an RPG that has “pick your pronoun” options, a thing which should not require any defense at all, because no one in their right mind would be upset about it, as it is entirely inconsequential to anyone who does not feel any concern about their own pronouns.

    You end up feeling like the only sane and logical course of action is to ignore these people, to let them enjoy their negative echo chamber, because fighting against them is too exhausting, too confusing, or too dangerous (depending on how outraged the mob becomes).

    But if you are a person who cares, and you feel badly for video game developers who are getting laid off because these hate campaigns can be quite successful at poisoning the gaming community against an otherwise good game, you want to speak up and fight back. There’s also the aforementioned problem that, if you are a fan of games, you likely want to interact with other fans of games (because it is always more fun to enjoy a game as a group), but online spaces for gamers are being taken over by this kind of aggrieved antagonist and every conversation ends up being poisoned by all these sorts of tactics. You can’t escape it, there is no place on the internet dedicated solely to the enjoyment of a thing anymore; subreddits for specific games usually end up being a place where people complain about it more than they celebrate it.

    So what can you do, really? If you fight back, you’re essentially just spinning your wheels. You’ll never get through to these gamers whose minds have been poisoned by negative content. You won’t convince a YouTuber whose livelihood depends on creating these hate-driven videos. And you’ll only be painting a target on your back for some internet loser looking to dox you. And if you do nothing, you risk sitting idly by while historic game studios are slowly dismantled by manufactured controversy.

    It’s unfortunate, but the anti-woke, anti-liberal, anti-kindness contingent of the internet has amassed a large amount of power, thanks to social media algorithms that promote their rhetoric to those most susceptible to it, leaving people on the other side feeling pretty powerless to fight back against it.

    For two solid decades, the internet was a safe haven for weirdos and outcasts to congregate and enjoy things together. Now, it’s an algorithm-driven hellhole—a bar full of angry, bitter people desperate to pick a fight. Worse yet, the bartender is paying them to do it because the fights keep people coming back.

    Why would anyone go to that bar?

    Why do I?

  • About two years ago I released a screensaver for macOS called Ealain, and versioned it v0.1 because it was not quite living up to my original vision for the screensaver. And while it has grown in popularity regardless, I always knew it could be better.

    Ealain is meant to be a macOS screensaver that is constantly creating new AI-generated images that grace only your screen. I wanted to power this functionality with the AI Horde, and have each screensaver generate its own batch of unique images.

    But it was easier and faster to essentially fake that functionality by having one person (me) generate the images periodically and upload them to a CDN, for the screensavers to download from. This was less cool, as every screensaver would be displaying the same imagery, but much easier and faster to develop, and results in a nice immediate gratification when you start the screensaver and images show up right away.

    However v0.1 had no local caching, so each time it displayed an image, it was downloading it from scratch from the internet. This made the screensaver something of a ticking time bomb for people on metered internet connections, and for the past two years it’s been bothering me that I put something so faulty out into the world, and I’ve been itching to fix my mistake.

    Without further ado, I am delighted to present Ealain for macOS v1.0, a version of the screensaver that satisfies my original vision for it, as well as expands the concept just a tiny bit.

    Here’s some samples of the kind of art you can expect to see in Ealain v1.0.

    Download it now over at https://github.com/amiantos/ealain!

    What’s New in Ealain v1.0

    AI Horde Generation

    Ealain v1.0 now generates images directly from the AI Horde, using the AI Horde Style system. This new setup allows me to have the screensaver generate images for everyone, but use the kudos supply from my worker to pay for the generations. If I ever run out of kudos, it’ll continue to work, it simply start generating images more slowly.

    You’ll be getting the same classic Ealain screensaver images (with a few changes to remove some generations I was never happy about) that you’ve been enjoying in v1.0, and now they will be unique for everybody. It’s essentially impossible that you will ever see an image that someone else has seen, which is pretty cool!

    Portrait Display Support

    Ealain v1.0 has another improvement, which is that it will generate unique art for monitors that are in landscape or portrait orientation. I have a computer setup with one landscape and one portrait display, and it’s always bothered me that the image displayed on the portrait display is just a ‘slice’ of the landscape image. Now you get full 9:16 aspect ratio images unique for that display, if you have one.

    Customizable Style

    Ealain v1.0 has gained one new feature, and that is the ability to override the screensaver style with one of your own choosing. This means that you can completely take over the screensaver and change the art style to something else. I’ll be thinking of and creating more styles that can be found in STYLES.md in the Ealain repo. Here’s one I created as an example, Cubism:

    That’s it! I hope you enjoy the screensaver.

    https://github.com/amiantos/ealain

  • In my early teens and twenties, I was a consummate blogger. Well, really, I was more of a diarist, spewing thoughts about my life nearly continuously onto a blog of some form or another. In the late 2000’s, I decided to use my ability to spew text to promote music I liked, by converting my journaling domain staires.org into a music blog. The original conceit was that I would post a song every day, for at least a year, complete with a personal anecdote about my personal connection to the song, or at the very least a couple paragraphs of my thoughts and opinions on it.

    I believe I made it up to around day 230 before I decided I could not keep up the daily pace and started posting less frequently. But that was a lot of time to do the daily posting consistently, and it got me some early attention. Eventually I got picked up by Hype Machine, which helped promote the blog further. They still have an archived list of 208 staires! songs that they syndicated, which is pretty cool to see. Over time, I built up a small following and made some lifelong music-loving friends along the way.

    But life, or mental illness, or monetary troubles got in the way and I eventually stopped updating it. I gave up. I guess I lost the love of the game? I forgot what the point was? I was going through a lot of stuff in the early 2010’s, so who knows what my exact reasoning was. In a bit of a snafu, when I cancelled my hosting for the website, the host ended up turning off auto-renew for the domain. When it came up for renew, they registered it for themselves and wanted to charge me hundreds of dollars to get it back, which I did not have. And that’s why the staires.org domain was advertising car parts and other junk for a long time, probably over a decade.

    A couple days ago, I randomly typed the domain into Squarespace, and it said it was available. I was shocked, and I bought it immediately, without thinking much about it, just glad to have it back with me where it belongs. But then I realized today, I’ve never stopped writing about music I like, recommending it to others, I’ve just been doing it privately, directly to friends or small community chatrooms. And in a lot of cases, when I do it that way, I’m just shouting into a void, and most people aren’t even paying attention. What’s the point of doing it that way?

    At least if I write a post on a website, it’s something random people can run into. People who enjoy the songs I post can add me to their feed reader, or subscribe via email these days, and I can go back to promoting the music I love to a possibly wider audience.

    So, without further ado, you can find me blogging about music again over at https://staires.org. Enjoy!

  • I noticed a lot of people ask in the AI Horde Discord how to run a worker. I thought it would be useful if the community had a blog post to point people to, and maybe that can be this post. Let’s go!

    How to get an AI Horde API key

    Be sure to get an AI Horde API key if you don’t have one. You can register here: https://aihorde.net/register. Be sure to protect this API key, put it in your password manager or somewhere else for safe keeping. This API key will be your ticket to all the kudos you can earn.

    How to Run a Text Worker (“Scribe”) on Windows

    If you’re using Windows, you’re in luck, because running a Horde worker is easier than ever thanks to Koboldcpp, which compiles all the bits and bobs into one executable file for you.

    Step 1: Download Koboldcpp

    Go to the Koboldcpp Releases page, and grab the executable most relevant to you. This is probably koboldcpp.exe. If you have a newer Nvidia GPU, grab koboldcpp_cu12.exe. Stick this file in a folder somewhere, like D:\koboldcpp\koboldcpp.exe

    I like to make a presets and models folder in here, so your folder might end up looking something like this depending on which version of koboldcpp you downloaded.

    Screenshot showing Windows Explorer in a folder, with "models" and "presets" folders visible and a single file named koboldcpp_cu12.exe in it.

    Step 2: Download a Model

    Koboldcpp can run models quantized in the GGUF format. What does that mean in the most basic sense? Koboldcpp can run models that are compressed in a way that allows them to run on lower-end hardware, with some trade-offs (like decreased quality of generations). For example, CohereForAI/c4ai-command-r-v01 requires ~70gb of VRAM in its original form, but a 4-bit quantization (available here) of it only requires ~23gb of VRAM.

    We’re going to focus on a model that everyone should be able to run locally, Sao10K/Fimbulvetr-11B-v2, which is a smaller model that really excels at roleplay chat, and there is plenty of demand for it on the horde. It’s a personal favorite of mine as well.

    Step 2.a: Find a GGUF version of the model

    Google (or use your search engine of choice) “<model name> gguf”, in this case we’ll look for “fimbulvetr-11b-v2 gguf“. Usually the first result going to huggingface.com is what you want. In this case, we’re going to end up on the second result, at mradermacher/Fimbulvetr-11B-v2-GGUF, because it gives us more options in regard to quantization sizes.

    Step 2.b: Download a GGUF version of the model

    You can see on the model page for that GGUF version, that there is a chart that tells you how “good” the various quants are, but here are some general tips:

    • Models hosted on the horde should be Q4 or greater, to ensure best generation quality.
    • You should pick the largest quant that looks like it can fit in your GPU’s VRAM.
      • If you have a 8gb GPU, get a model under 8gb, etc.
    • If you aren’t sure what to try, try Q4_K_S first. You can always download a bigger quant and try it later.

    Download the .gguf file into your koboldcpp/models folder. My models folder looks like this:

    Step 3: Configure Koboldcpp

    Now we’re ready to launch Koboldcpp and start configuring it. The initial Quick Launch screen has all the main information we need to worry about.

    1. Hit browse and pick the .gguf file you downloaded into your models folder.
    2. Set the context size you want. I like to have plenty of room for context, so I pick 8192 by default, but if you have less system power you should try 4096 to speed up generation times.
    3. If you’re certain the model will fit entirely into your VRAM (because you downloaded a model smaller than your available VRAM), set a large number here. If you don’t know if the model will fit into your VRAM, there will be a section below about figuring out layer quantity.
    1. The model name is very important to get right, as it determines how much kudos you will get for your submitted generations. If you put an incorrect name here, you will get very few kudos. To makes sure you put the right name, check the model whitelist for the name of your model, without any quantization naming attached. In this case, we see that the base model of Sao10K/Fimbulvetr-11B-v2 is in the list. But that isn’t the name we want to put into the “Horde Model Name” slot, we want to identify that the model is being run with koboldcpp, so we put in the model name as “koboldcpp/Fimbulvetr-11B-v2”. Nothing else matters, you do not need to include the quantization level in the name.
    2. This is where you put your API key so you can receive kudos properly!
    3. Your worker name should be the same every time you run your worker, regardless of the model being used.
    4. Save your configuration to a .kcpps file so it is easy to reload it later. This is what the presets folder is for, save your configurations in there so you can easily use them later on. ⚠️ Koboldcpp doesn’t save your settings automatically!

    Once your configuration is saved, hit Launch!

    Step 4: Is it Working?

    Once your worker is running, you should end up with a terminal display that shows you what your worker is up to. It looks like this.

    1. This is how many kudos you are earning for this specific job. If this number is very low, like 1 or 2 kudos, you likely have your model name configured incorrectly or the model is not whitelisted for the horde. Double check that your model name is correctly entered and that the model is whitelisted.
    2. This read out shows how many jobs your worker has completed and how many kudos per hour you are making. If you’re not earning several thousand kudos per hour, your worker is likely configured incorrectly or you’ve picked a model you are not able to run at a decent speed.

    You can test your worker through the horde by using Kobold Lite. Load up the website and then click on the “AI” button in the top left corner.

    1. Put in your AI Horde API key if you haven’t before.
    2. Check the box “Select By Worker”
    3. Look for your worker in the model list. If you don’t have a Purple Heart here, it’s just because your worker is not yet “trusted”. More on that later.

    Click OK, then pick a desired scenario (KoboldGPT is the easiest for testing, I think) and submit a chat request. You know your worker is running when you get a message back and it shows your worker name and model name in the bottom of the client, like this:

    If you ended up here and all looks well, congratulations, your worker is running and racking in the kudos.

    Troubleshooting

    “Where are my kudos? I’m not getting the right amount of kudos.”

    To prevent abuse of the horde by bad actors, when you first start running a worker, half of your earned kudos are held in escrow. After a week or two of running a worker without issue, you’ll become “trusted” and receive all kudos owed to you (the kudos held in escrow plus all future earnings). (By “without issue”, I just mean that your worker is returning proper generations and there is no monkey business happening.)

    “What was all that about layers and how can I run models larger than how much VRAM my GPU has?”

    When you are configuring Koboldcpp, it asks you how many GPU Layers to use. This can be useful if you want to run a model that is just slightly too big for how much VRAM your card has. But how many layers does each model take up? There is a way to guess this yourself, but I like to just try to load the model in Koboldcpp and see what it says. For example, the 8bit quantization of Fimbulvetr I’m using displays this in the console when loading.

    You’ll see it says that this model had 49 layers that it is loading in the GPU memory. If we didn’t have enough memory to store the full model, we could configure Koboldcpp with GPU Layers set to 40, and then it’ll load most of the model to the GPU, and the rest to system memory for the CPU to use. This can be very slow, so it’s recommended to only offload as many layers as needed to run the model at all. The more layers on the GPU, the better!

    “I need more help!”

    Not a problem at all, there are a lot of people who are willing to help you over on the KoboldAI Discord. There are lots of people far more knowledgable than I am, and if you decide you like running a worker a lot, they can help you level up your hosting game. Remember, this is just the most basic guide, there are more robust methods (like aphrodite) that allow you to run several models at once if you have the hardware capable of doing it. Feel free to ask about that in the Discord.