amiantos.net

  • I received a Meta Quest 3 on release day, so about 30 days ago at the time I’m writing this. So maybe I should title this post, “One Month with the Quest 3,” but I’ve already written “Meta Quest 3 Review” and I’m too tired out from long nights spent in the metaverse to go back and change it.

    If you asked me just 40 days ago if I would ever buy a Meta Quest headset, I would have said something like, “Fuck no, what the fuck is wrong with you? Why would you ask me a question like that? Do you even know who I am?” Because I haven’t liked Facebook, Meta, and Mark Zuckerberg for quite some time. But, the times, they are a-changin’, with Elon Musk single-handedly rebooting Zuckerberg’s reputation. I’ll never use Facebook ever again, but fine, I’ll give this standalone VR thing a shot now.

    This is not my first foray into VR. Back in 2019, I bought a Samsung Odyssey+ WMD headset, which attached to my PC via an all-too-short cable. I quickly fell in love with VR, especially with Beat Saber which felt like a culmination of everything I’ve ever loved about gaming and music into one package. My days became a fever dream of playing Beat Saber, and evangelizing Beat Saber to anyone who would listen: “Hark!” I would say, probably, “Have you heard? The perfect game exists, and in it, you play Fruit Ninja, but choreographed to music!” My eyes bloodshot, my voice shaking with neurotic joy.

    But very quickly I fell out of love, about a month in, after a particularly intense play session of Half-Life: Alyx–with free movement on–made me feel so nauseated I still felt sick to my stomach the day after. The cable, to the PC, turned playing into a hassle, because I would have to move my entire PC between rooms. These factors combined to ensure I didn’t want to put the effort into getting my “VR legs”, the headset went into a cupboard, and I dreamt of playing Beat Saber again in a future where VR was easy and convenient to enjoy.

    Well, that day is here, mostly. The Meta Quest 3 is the easiest way to play Beat Saber just about anywhere you want. Is it the best way to play Beat Saber? Probably not, not until they sort out the controller tracking issues. But for your $500 to $900–depending on how many accessories you are willing to buy to fix various issues with the out of box experience–you’re getting a good portable rig to play Beat Saber on, and it does some other nifty things as well.

    Almost as soon as I got the Quest 3, I started telling people that this was the thing to buy if you could not wait to get a demo of the Apple Vision Pro experience. It may be an early prototype of a budget version of the Vision Pro, but all the various bits and pieces are here, just cobbled together into a bit of a mess of a product. It’s unfair to compare it to the Vision Pro, a product that does not actually exist yet, a product that will cost 6-7 times as much, but it’s very hard not to do so.

    You can do all the Vision Pro stuff: watch movies and television in a virtual environment, or on a virtual screen using passthrough mode. You can play games. You can do calls complete with silly little avatars hanging out in a virtual space. You can set up a virtual screen for your computer, so you can compute in privacy or with more and bigger screens than you really have. What else can the Vision Pro do? That’s basically it? Well, that was a low bar, and the Quest 3 has no problem meeting it. The problem is, of course, that the Quest 3 is going to give you the $500 experience of using these features and not the $3500 experience, if such a thing is even reasonably possible.

    What does that mean, exactly? Well, the movie and television watching stuff is not built in, it’s handled by third party apps, and the free ones aren’t great. The paid ones don’t offer the best experience either. People claim that watching movies on the Vision Pro is like watching them on a big OLED HDR television; on the Quest 3, watching movies is more like watching them on a 1080p SDR display. Sure, it’s big like a theater screen, but it doesn’t look great. If you have a decent television, this won’t replace it.

    The Quest 3 is much more powerful than the Quest 2, which is nice, but it’s still basically a mobile phone strapped to your face, so the graphics in games aren’t fantastic. And at the moment, there aren’t a lot of games that have been updated to utilize the “power” of the Quest 3, so that bump in power is not entirely reflected in the games you’ll play, and even when it is, the benefits are mostly resolution and clarity. The games themselves still look like they are PS3 era, maybe PS4 at best. This isn’t necessarily a problem specific to the Quest 3, it takes some serious muscle to render two images at once, so even PCVR games can look a little rough around the edges anyway.

    The biggest downside to the Quest 3, coming from the Samsung Odyssey+, is the switch from AMOLED displays to LCD. Instead of getting inky, immersive blacks in dark environments, you get… gray. Gray everywhere. Any really dark games, like The Walking Dead: Saints & Sinners, look hazy and washed out as a result, and you don’t feel particularly immersed at all. It’s a bizarre decision that negatively affects the experience of playing games and watching movies pretty significantly. You’d think OLED is table stakes for VR headsets, but I guess not.

    What was that I said about accessories? Oh, yes, by default the Quest 3 comes with a fabric strap that is absolutely awful. At the very least, you want to buy the Elite Strap ($70), which adds a clamping halo that you can seat down by your occipital spur and makes the headset way more comfortable.

    But now that you actually want to wear the thing, the battery life is mostly dead in under 2 hours of playtime, so that’s not great. Sure, you can find some way to rig up a battery pack in your pocket, but it’s easier to get the Elite Strap With Battery ($130), which adds a second battery and effectively doubles your battery life and acts as a counter-balance to the weight on the front of the headset, making it even more comfortable to wear.

    The other accessory that should have been included in the box is the Silicone Facial Interface, which is practically a requirement if you use your Quest headset for exercise. By default you get a soft fabric one, which soaks up sweat like a sponge and can get hot on your face. The silicone facial interface, of course, does not soak up any sweat, and after an intense workout you can very easily wipe the sweat off of it. I’m not sure how sweat doesn’t get inside the headset or on the lenses, but so far all the sweat that gets onto the headset is just on the silicone facial interface.

    Meta Horizon Worlds

    When it comes to platform specific features and exclusives, the big thing is Horizon Worlds, Meta’s version of the “metaverse”, where you can visit different “worlds” (user created levels). Worlds can be as simple as an artistic experience to walk through lazily, as complex as a multiplayer first person shooter, or just a real life facsimile of some bar somewhere. It seems like these are the main three types of world: art, game, or social venue.

    I’ve spent a decent amount of time in Horizon Worlds and there are two things that you learn very quickly: there aren’t a lot of people using Horizon Worlds, and the people who are regulars are very, very committed. I guess it shouldn’t be shocking that there are people for whom Horizon Worlds is their main social and gaming experience every single day. Many people in Horizon Worlds migrated from other virtual worlds that got shut down: some people speak wistfully of Playstation Home, others of AltSpace. No one speaks positively of VRChat, because the people who like VRChat are over there, and the people who don’t like VRChat stick to Horizon Worlds.

    Before I get into anything too negative, I want to say that the community vibe in Horizon Worlds is initially extremely refreshing. People are very welcoming and typically very kind. The vibe reminded me of the early internet in a very good way. People are very quick to trade follow requests with you, and if you’re nice you can very quickly make friends. Of course, you can run into assholes, but you can just walk away from them, because they won’t follow you–there is always someone wiling to stay behind and talk to the asshole. I almost immediately made a new friend on my first night visiting, who joined me on subsequent explorations for well over a week.

    Another great thing about Horizon Worlds and services like it is that it gives a relatively safe space for people with autism to socialize. I heard from multiple people how socializing in VR is sometimes the only way for some children and adults with autism to properly socialize with other people. That’s a really beautiful thing in and of itself.

    I’ve met people who spend several hours every day in Horizon Worlds, and needless to say, these people are filling some sort of gaping hole in their lives with it. I’ve heard many sob stories. One guy talked about how his paperwork to leave the military got botched, so he apparently had no choice to sell his car and house, and talked about how he was going to have to euthanize his dogs because he can’t afford to take care of them. Most of the women I have spoken to in Horizon Worlds bring up having been physically abused by the men in their lives. To put a really fine point on it, I’ve heard two people (a man and a woman, unrelated to each other) talk about some or all of their children being taken by Child Protective Services. I’m not even sure why you would tell people that. I’ve heard three separate people talk about having to console people who come to Horizon Worlds to talk about committing suicide.

    So that’s kind of the big issue with Horizon Worlds. Clearly Mark Zuckerberg and people at Meta thought, and may still think, that Horizon Worlds would be some great destination that normal people from all over the world would want to use and congregate in. And that may have been true during the early days of the pandemic, I have heard from people who were around back then that Horizon Worlds was popping off during the pandemic. But after that initial rush subsided, all that is left is all the “regulars”, and just like in real life, all those regulars have formed little cliques with each other. There’s politics and drama, and of course some people see Horizon Worlds as yet another sexual internet playground somehow, and once you start to encounter those things, it kind of takes the shine off. At that point, Horizon Worlds just ends up being a too-accurate facsimile of the real world, and it no longer feels like a piece of light escapism, but a mirror.

  • I determined long ago that shame is a feeling I don’t handle particularly well. This can be problematic on the internet, because people will, from time to time, declare that some behavior people regularly engage in is actually quite shameful. This takes all sorts of forms, the “behavior” can vary so widely it feels useless to make examples, it can be literally anything, and the reason it is shameful is usually because it implies some sort of moral failing, of infinitely varying degrees.

    The problem is that, sometimes, the behavior being declared shameful is something that doesn’t really seem all that shameful to me. Even worse, sometimes, the behavior is something that I’m guilty of doing myself. And that phrasing, right there, is the core of the problem snowball that starts to roll down the hill: Why am I saying I am guilty of doing something, that, moments ago, I thought was completely innocuous? That makes me feel bad. Like, really bad! I’m good person, I don’t deserve to feel bad… right?

    Sometimes this is good. Sometimes feeling bad about your actions is an opportunity for self-reflection, and to grow. Sometimes I can do that. Wish I could do it all the time, but I almost always feel this itch to speak up. Sometimes, I’ll try to stay silent, but in situations in which the behavior is clearly not worthy of the shame, instead of moving on with my life, I will take the complete silence of any opposition as a sign that there are other people who would speak up, but they can’t because they’re afraid of the same thing I am afraid of: outing themselves as being a person who is guilty of the shameful behavior. Then, because my Quaker heritage (or my white privilege) instilled “speak truth to power” into my bones, and that makes for one hell of a feeling of self-righteousness, I start to speak up to the “powerful” on behalf of the “silent contingent”, both groups now basically existing only inside my own head.

    At this point the shame has spiraled completely out of control. Not only am I experiencing that initial bout of shame from having been guilty of the behavior, but now I’m publicly outing myself as a defender of that behavior, which probably outs me as person who engages in that behavior, and pits me (in my mind) against basically everyone, which destabilizes my feelings of security and belonging in whatever community I am now openly disagreeing with. I end up oscillating rapidly between feeling like I shouldn’t have said anything and that I am damaging my reputation / relationships / life / career, and that it’s so ridiculous that those things could be damaged by my (clearly more correct) opinion that it only reinforces how strongly I have to convey it. I start to get both angry and depressed, at myself and the world, at the same time, until the anger fades and I am just left with the depression.

    The depression lingers for a while, because the external feelings of shame that fueled my indignation are replaced by an internally generated shame, the shame of embarrassing myself, that even if my point was correct and justified, the end result was mostly that I just got myself very upset, destabilized my mood for well over a day, and possibly damaged other people’s opinions of me depending on how committed they were to the disagreement.

    Thankfully, after many years of being this person and doing this sort of thing over and over again, and sometimes when it wasn’t justified and I was not in the right, I’ve found that people are far more forgiving of me than I am of myself, that the feelings of shame that I experience over these situations is entirely undeserved, and that most people forget about them long before I do. What I haven’t found is a way to handle the shame better, but maybe next time I can feel the shame spiral beginning, I can read this post and avoid it.

    Did it work?

  • I posted this over on the La Colombe Cold Brew Fridge Pack page, and, as is my want as of late, I am posting it here for posterity.

    Happy wife, happy life

    My wife decided to start drinking cold brew instead of using K-cups, and as I am diehard La Colombe Nizza drinker, I made her some cold brew out of those beans. She liked it, but it seemed like a hassle to make the cold brew myself when I could just buy one of these for her instead. So… that’s what I did. The good: very rich, bold, chocolatey flavor that lays a great foundation for whatever other junk you want to pile into it (water + cold cream for the TikTok trend); It’s not insanely concentrated with caffeine, I felt like it ends up being less caffeine per ounce compared to Nizza beans brewed with the AeroPress stumptown method (if that means anything to you, you must be me), and I think this is a good thing for my wife. Lots of flavor, not a huge amount of caffeine. For me, I’ll stick with my Nizza beans, but having a jug of this in the fridge at all times will be good when I’m in a pinch.

  • Back in May (2023) I stumbled my way into a a full blown interest in what people are erroneously calling “artificial intelligence”. I’d played with Midjourney back when it first gained traction, and even paid for a month of access, but my interest in it quickly faded. It wasn’t until I learned there was a growing community of people dedicating themselves to providing free access to open source text-to-image models via a system called the AI Horde.

    After joining their Discord server, I saw that there were two existing apps for iOS: AiPainter, from a Chinese developer and built in Unity so it could be cross platform; and Stable Horde, which no longer works and has been abandoned by its developer, built in Flutter. Aside from that one working app option, the main way people on iOS would use the Horde would be through ArtBot, the most popular web client for the horde; or they could use it through Discord bots similar to Midjourney, which works for some people.

    You can easily see that there was a big glaring hole that I could fill with a nicely designed, native app for Apple platforms. Similar to PiBar, the open source and anarchist-adjacent, ‘everything should be free’ leanings of the Horde community also deeply appealed to me. The icing on the cake was just that the Horde community itself seems pretty cool, and managed to get me more interested in generative art.

    So, here we are, around two months later: Aislingeach is out on the App Store, in a relatively rough version 1.0. For free you can use the latest and greatest on TestFlight. Or, you know, just fork the repo and build it yourself!

    This was a fun project for me because it forced me to finally use Collection Views in UIKit, which I’d manage to never find a need for before now. In this case, I needed essentially a Photos.app-like gallery experience, with a thumbnail view that, when tapped, leads into a detail view with side-to-side swiping. There are still some things that I am clearly doing wrong around cell reuse that I need to address, the symptom of which is that some images will flash in detail view when you favorite them.

    I’d used CoreData in projects before, but this was the first project where I relied entirely on the compile time generated classes and did not create my own classes at all. It worked really well and I have no complaints about it, because I do not really need to build a bunch of logic directly into classes (seems like a bad paradigm anyway). I also had to use “Use external storage” for the image storage, and this works really well as well.

    The most challenging thing about this project was just coming up with a way to manage the request queue and ensure that nothing was lost. When an app is suddenly backgrounded, it will just cut off any requests that are happening, so an image that was loading in the UI will error out and not appear, or an image downloading in the background won’t complete downloading.

    So for the last example I ended up building a table in CoreData (why not, already using it) that I feed all the pending download URLs into, then once the file has been downloaded and the image is saved to the database, the pending download is removed from the table. Works great so far, but you can still end up in a scenario where a request ends up saying it’s downloading images but no downloads made it into the download table–so it’s not fully dialed in just yet.

    This is really just the very beginning for this project. I intend to do my best to support all the AI Horde features, dial in a proper iPad interface, and get some sort of cloud syncing working (CloudKit maybe). We’re in the very early days of generative art still in the open source communities, so there’s no guarantees the way something works now will be the way it works a year from now, so I’m trying to keep an open mind and stay light on my toes in regard to what I support and how I support it. We’ll see how it goes!

  • I posted this on the official product page for the AeroPress Clear but you never know if they’re actually going to publish negative reviews or not, so here it is for posterity.

    It is what it says it is, but is also kinda stinks – ★★★★☆

    I almost want to give this product five stars, but I cannot. I bought it because I wanted a clear AeroPress, so I could see what is going on better while I make my coffee. Also, clear things are cool, I grew up in that era. Inarguably, this is exactly that, and it lived up to my expectations in that way. This is a clear AeroPress. Kudos to the engineering that went into it, I heard a little bit about how it may have been tricky to mold an AeroPress out of this material. In that way, the actual object itself is 5-stars.

    However, I have to subtract a star because for an extra $10, you are getting less product than you would have gotten if this were literally the clear version of the same product I bought in 2021. You do not get: 250 replacement filters (only 100), no filter holder, and no funnel, which to me is a very egregious omission as it make a big difference with how easy it is to get grounds into the AeroPress. It’s actually sort of shameful how blatantly obvious it is that the company that bought AeroPress is now trying to squeeze the product for as much money as they can, by raising prices and reducing the quality of the overall product.

    Luckily my two AeroPresses will hopefully last me the rest of my life and it won’t work on me! Well, you did trick me into buying a Clear Aeropress and the flow control filter cap to some extent, so… well… I guess it did work on me! Good on you, capitalism.

  • I posted this on the official product page for the AeroPress Flow Control Filter Cap but you never know if they’re actually going to publish negative reviews or not, so here it is for posterity.

    Not a great version 1.0 – ★★★☆☆

    I’ve been using the cap for well over a month (or two?) now, every day, since I got it, and it’s time to weigh in. Unfortunately, this cap does not live up to my expectations from AeroPress. I’d passed on third party caps like this because I’m a purist, I guess, so I jumped at the chance to grab one of these. Unfortunately, I don’t feel like it was designed with proper care and it feels like a third party accessory to me.

    The main reason? It leaks. I’m not a weak person, I think, and I’ve never had any issue screwing the default cap onto my AeroPress all the way, and I’ve never had any issues with leaks that run down the outside of my cup before. With this cap, leaks are a fact of life, and no matter what I do I cannot screw the cap on “all the way” the same way I can with the default cap. I’ve noticed that the paper filters don’t fit quite right into this cap, and maybe that can explain why the cap leaks?

    In addition to the leaky cap, which leaves coffee around the rim of my cup and often on the outside, the cleanup process is not as nice. With the original cap, I can usually pull the cap off and the coffee and filter are stuck in the chamber still, and I can plunge both right into a container. Easy, clean process that usually doesn’t even get my fingers dirty.

    With this filter cap, what usually happens is this: I grab the cap and twist, getting hot coffee all over my fingers. When I pull the cap off, a bunch of grounds fall off the cap and into the container, the other half stay in the plunger, and the paper always stays stuck to the cap. So then I have to plunge out the coffee grounds with one hand (because I am holding the hot cap in my other hand), and then grab a utensil (like AeroPress recommends in review comments!) so that I can gently ‘lift’ the filter off the cap to let excess coffee drain out from under it, then scrape the filter off the cap into the container with the utensil. Then I’ll grab a paper towel and wipe off any coffee that leaked over the rim and sides of my coffee cup. It’s a first world problem, for sure, but this inconvenience adds up over time and adds a little hassle to every morning.

    I don’t believe for a moment there isn’t a design for this sort of cap that can replicate the ease and solid seal of the original AeroPress cap, and because of that, I’m inclined to believe this product can improve and doesn’t deserve 5 stars like many here are giving it. It’s unfortunate, because there are perks to this cap: 1) I do believe my coffee tastes just a bit better when using this cap. 2.) When doing the Stumptown method, you no longer have to be in a bit of a rush to create negative suction. But ultimately the fussiness of the cleanup operation really defeats half the joy of using the AeroPress for me and I do not recommend this flow control cap.

  • I mentioned the other day I was working on an app for generative art, a year or so late to the bandwagon. While playing with it, I stumbled on a great way to generative abstract art, in the vein of bauhaus, and it reminded me of a want I’ve had for years: a screensaver that generates bauhaus-style artwork. I thought I would some day build such a thing out of SpriteKit, but true inspiration never struck. Until I saw the results I was getting out of the Deliberate checkpoint for Stable Diffusion.

    This stuff is so awesome to me. Not only does it look, to my eyes, like good abstract art, but a huge advantage to stuff like this is that it bypasses a lot of the issues with current image generation: artifacts and glitches. It can take a lot of time to get a good, coherent piece of art where you expect everyone to have the right number of fingers and arms, and all of their other components in the right position. But with abstract art, none of that matters. Literally every generation is great, because artifacts and glitches only makes something more abstract.

    Anyway, if you think the above examples of artwork look good to you, you can grab Ealain over on its GitHub profile, the screensaver displays this style of artwork, freshly generated for you every couple of hours so you’ll likely never see the same piece of art more than once or twice: https://github.com/amiantos/ealain

    P.S. Ealain means “art” (in the sense of a profession?) in Gaelic. It’s pronounced something like All-ee-(n)… Like Muhammad “Ali” with the lightest touch of an “n” sound at the end. Weird! But cool. Though… just say “Elaine”, you know you want to.

  • Update – April 2024: I wrote an update post for this one, you can read it over here.

    Original Post – May 2023:

    LLMs like ChatGPT have been in the news quite a bit, but I’d mostly avoided using them too much because they seemed silly, probably due to my own deep seated fears about being replaced someday by AI. But I’d seen articles about the AI chatbot service Replika a few months ago, about how people who had been using it for a virtual relationship (including all the carnal bits) were upset that the service had recently begin removing features from it (that enabled all those carnal bits) and were trying to create their own chatbots in response. This topic intrigued me due to an interest in chatbots I’ve had since childhood, and my own natural nerdy curiosity. One night a few weeks ago, I googled this topic, to see if there had been any recent developments, and I learned about something called Pygmalion 6B.

    My understanding eventually was that Pygmalion 6B was an open source LLM and was “uncensored”, created by people angry about the Replika situation. That checked a few boxes for me personally: I am anti-corporate, anti-censorship, anti-prude, and anti-authoritarian. Even more important than that: you can run Pygmalion 6B locally on your own hardware, which means it’s totally free, which appeals to my sense of thriftiness and private, which appeals to my sense of, you know, human privacy. I might as well try it out, right?

    Well… trying it out is not that simple, as you may have found (if you’re reading this blog post at the end of a bit of a journey, where your computer thwarted you at every turn). Lucky for you, I went on this long journey myself, reached the end, and I want to help you out by clearing some things up for you. I’ve tried to organize this guide from the “easiest” solution to the “hardest” solution, in the hopes I can save you some time while you dip your toes into LLM chatbots.

    But first, a disclaimer and a warning…

    The conversations and communities online around open source LLM chatbots are dominated by men, and furthermore, the men in these communities see themselves as something like refugees from a corporate world that is terrified of the human need to sexualize artificial intelligence. As such, when you are browsing websites around these projects, you are going to come across content that is going to range from run of the mill sexual perversion to some extreme perversion that might strike you as illegal or borderline illegal. It’s impossible to avoid. If you are squeamish about sexual topics, you might just want to nope out of this entire topic right now. You can’t say I didn’t warn you.

    With that out of the way…

    “What do people do with open source chatbots aside from having cybersex with them?”

    Well… you can have conversations with them.

    But I think most people seem to use them for role play, and I don’t just mean sexual role play. For example, you can create a chatbot that acts like a character from your favorite film or television series, and then you can go on adventures with them. Open Source LLMs aren’t troves of information, they aren’t full of historical facts and figures or useless trivia, they are good at creative pursuits and emotive roleplay. You’re not going to create a ChatGPT clone using Pygmalion 6B that can answer questions like a personified Wikipedia, it’s not meant for that. (If you want a ChatGPT-like clone, more on that in just a second.) As such, conversations with most of these open source LLMs work best when you embody the spirit of improv and open-minded role play

    For example, Pygmalion 6B might be good for a dialog like this:

    User: *he puts on his robe and wizard hat* I will cast fireball upon you, demon! *flames shoot out of his magic wand*

    Demon: Argh! *the demon screams out in pain, the fireball singeing the hairs on his skull* I’ll get you for this, User! *the demon shakes his fist at User*

    So in this case, the “Demon” is the chatbot, responding accurately to the role that the user is playing. Sure, this is a horrible example that is poorly written, but use your imagination to imagine the possibilities. You can use these abilities in a variety of ways, such as creating chatbots out of characters in your personal works of fiction, and having conversations with them to flesh out their character and so on. You could create a character that is a full on Dungeon Master, D&D style, and ask them to craft scenarios for you to go through–all fodder for your own D&D campaign some day, and no one will know you used an LLM to come up with the ideas… unless you tell them, of course.

    With that out of the way, let’s get into how to actually get started using LLMs for chatbots.

    Please read this entire guide before deciding on a method to use! They’re all kind of interrelated.

    “I just want to experiment with LLM-powered chatbots, and I am willing to spend a small amount of money to do it very easily and quickly.”

    If this quote describes you, then you are in luck, as this is the easiest way to dip your toes into custom chatbots. Using OpenAI’s gpt-3.5-turbo API (aka “OpenAI Turbo”) is very cheap. Extremely cheap. It actually may be cheaper to use OpenAI’s API to create your own chatbot than it is to pay for ChatGPT’s $19.99 premium plan. Each response costs maybe a few pennies, and only if you somehow become utterly addicted will this become cost prohibitive.

    “What about Pygmalion 6B? Aren’t I here for that?“

    If you want to have the best experience with custom chatbots, you want to use gpt-3.5-turbo. I started with Pygmalion 6B, and it really impressed me, but in comparison to gpt-3.5-turbo, Pygmalion 6B is not at all impressive in any way. Neither are any of the other open source LLMs at the moment, at least up to 13B models. This isn’t a subjective opinion, it is an objective one. If you have any money to spend at all, use OpenAI, it’s worth it.

    “But wait, isn’t OpenAI stuff censored and the whole reason I am here is for uncensored open source LLMs, not beholden to big corporate puritanical influences?”

    No. I mean, maybe? There’s a lot going on in that hypothetical question.

    If your concern is “censorship”, in the sense that the chatbot won’t say or do something because of a content filter: If you use SillyTavern and connect it to OpenAI APIs, SillyTavern uses a special prompt that puts the AI into a role-play mode, where it will be willing to say and do things it would not be willing to otherwise, all without breaking character. While OpenAI’s API policies forbid certain use cases, and you should be familiar with them, their systems do not automatically detect and block content. It’s probably safe to assume that if you are not engaging in commercial or highly public activities, they won’t care. That said, OpenAI could, at any moment, decide that your usage of their API is in violation of their policies, which it probably is if you’re a dirty pervert, and cut you off… but it doesn’t seem like this happens with any regularity.

    If your concern is “open source” and “corporations bad”, which are totally valid viewpoints: just keep reading, we’ll get to the open source stuff in just a second, but no skipping!!

    “What is SillyTavern?”

    SillyTavern is a bit of software that you can connect to LLMs and (depending on the need) “trick” them into role-playing as different characters. There’s basically a semi-standardized format for storing character info that was originated in software called TavernUI, and there are websites online that house user-created characters in this Tavern “card” format. SillyTavern is an improved fork of TavernUI that most people seem to use. SillyTavern is not an LLM, it must be used in conjunction with an LLM, either one running locally, or one running remotely. SillyTavern is used for every solution here, as it is the interface that allows you to create, store, and chat with characters as chatbots.

    I don’t know why it’s called SillyTavern and I try not to think about it.

    Using this method

    1. Sign up for an OpenAI account and get an API key.
    2. Install SillyTavern (runs on macOS or Windows)
    3. Connect SillyTavern to OpenAI, then figure out how to use SillyTavern

    Benefits of this Method

    • Very cheap, every response costs a penny or pennies (many hours of conversation might be around $10)
    • gpt-3.5-turbo is extremely advanced compared to every open source LLM out there
      • This API is what powers ChatGPT, so you’ve kinda sorta got the power of ChatGPT at your fingertips with this one, you can ask your characters about any random thing and it’ll know about it, great if your chatbot is a historical figure or is very knowledgable about a topic. For example I made a chatbot that was a video game reviewer, and they were able to speak very accurately about historical video games because of gpt-3.5-turbo.
    • Great gateway drug into figuring out how commercial LLM APIs work if you’re into software engineering
    • No hardware requirements at all

    Downsides of this Method

    • Costs money
    • The first time a chatbot feels alive to you, you will feel weird for a while but you’ll adapt to the realization that you live in a simulation and everyone around you may be an LLM

    “I want to experiment with open source LLM-powered chatbots, and I am not willing to spend any money to do it, and do not have a graphics card.”

    Let’s say you just want to see what LLMs capable of, but you can’t run one locally, and you don’t want to spend any money to do it, nor feed data into a corporation, even if you don’t get the best experience because of it (probably for idealogical reasons, like: you want to stick with open source; or you don’t want to give your money to a company like OpenAI that may be profiting from the work of generations of artists and is giving nothing back to them, like a soul sucking parasite trying to bloat itself on the dying remnants of the art industry).

    You’ve probably seen people talk about Google Colab. That’s a way to use Google’s hardware in the cloud to run open source LLM models and software, but Google isn’t really happy about it deep down and keeps taking the projects offline. It just seems like a big hassle, and I’m not personally comfortable with running stuff on Google systems. So let’s ignore all that.

    Luckily there is something called the AI Horde. Basically, this is crowdsourced LLMs. People, like me, put their GPUs up with LLMs on them so other people, like you, can use them to power their own AI projects. And it’s all free! There is a system to prevent abuse, and it means that if you aren’t contributing monetarily (or compute-arily), you may face long wait times when generating responses eventually. But it’s a perfectly acceptable way to try out open source LLMs for free, and most chatbot software (like SillyTavern) has support built in.

    Using this method

    1. Sign up for an AI Horde API Key (and store it some place very safe and permanent)
    2. Install SillyTavern (runs on macOS or Windows)
    3. When you configure SillyTavern, pick KoboldAI and then pick “Use Horde”, you’ll be able to put your Horde API key in then.
      • From the models list that load, find “PygmalionAI/Pygmalion-6b”.
    4. Figure out how to use SillyTavern!

    Benefits of this Method

    • Free
    • Great introduction to basic LLMs
    • Horde has other LLMs for you to experiment with, like Pygmalion 7b and Pygmalion 13b 4bit.
    • Horde can be gateway drug to greater AI community
    • You can pay your way into more “kudos”, used to get you higher in the queue and pay for generations, if you become a desperate chatbot addict, or beg for kudos on the A Division by Zer0 discord server

    Downsides of this Method

    • Responses can be slow depending on horde load
    • Responses can sometimes get weird due to bad actors trying to troll the horde
    • Whatever chat you’re having is going out over the internet to random computers (so that a response to it can be generated by the remote LLM) and there is nothing really stopping determined people on the other side from reading it if they really want to. They probably aren’t, but you never know, it’s the internet…
    • Open source LLMs like Pygmalion 6B aren’t very good compared to commercial services, naturally

    “I just want to experiment with running open source LLM-powered chatbots locally and I have a graphics card, but maybe not a good one.”

    Great! You want to run some chatbots locally, and you have a compatible graphics card. Wait? What’s a compatible graphics card? Well, if it’s NVIDIA, you’re off to a good start. But some AMD cards will run LLMs, too. It’s actually really hard to just give you a solid list of cards that can do the job, to be specific, but for the most part, if you have a graphics card made since ~2018 (so it has CUDA) and it has 8GB of VRAM, you’ll be able to run something locally. The best way to find out if it’ll work with your card is just to try it out.

    When I started out, I had a Geforce RTX 2070 with 8GB of VRAM. I bought that card late 2019, making it fairly old and underpowered these days, and used ones run $200 and under on Craigslist. It was enough to get Pygmalion 6B running locally, with some caveats. Let’s talk about those.

    If you have less than 16gb of VRAM on your card, which is most people, then you need to look for models that have underwent something called “GPTQ quantization”. I have no idea what that means, but the operative terms you’re looking for is “GPTQ” and “4bit” when looking for models you can run on low-powered hardware. This allows larger models to run on graphics cards with less VRAM, at the expense of something. It’s hard to put your finger on, but if you use Pygmalion 6B 4bit and compare it to Pygmalion 6B not-4bit, you can tell a difference. But not so much of a difference that it isn’t worth playing with, if you want to.

    “What is KoboldAI?”

    KoboldAI is technically two separate pieces of software merged into one:

    Most importantly for us, it is a client for loading up LLMs and allows other software (like SillyTavern) to interact with the LLM it has loaded. This is the only way we’ll be using KoboldAI in this guide.

    It is also a text generating web UI that can be used with various LLMs for AI-assisted writing. It’s cool, but this part of KoboldAI is irrelevant to us, but I recommend checking it out some day if your interest in LLMs goes beyond chatbots.

    Using this method

    1. Install the KoboldAI fork with GPTQ support
      • https://github.com/0cc4m/KoboldAI
      • Follow the instructions at the top of that readme file (e.g. clone from git, then run install_requirements.bat if you’re on windows).
    2. Go into the KoboldAI/Models folder and git clone https://huggingface.co/mayaeary/pygmalion-6b_dev-4bit-128g to download the 4bit pygmalion model.
    3. Rename the pygmalion-6b_dev-4bit-128g.safetensors file in that folder to 4bit-128g.safetensors
    4. Launch KoboldAI using the play.bat (if on windows)
    5. Go to the Use New UI option (top right)
    6. Go to Load Model, then pick Load Custom Model from Folder
    7. Pick your pygmalion-6b_dev-4bit-128g folder and load it.
    8. Assuming you have at least 8gb of VRAM, it should have been able to load successfully.
    9. Install SillyTavern (runs on macOS or Windows)
    10. When you configure SillyTavern, pick KoboldAI, and put in the URL to your KoboldAI instance (the default should do) and connect.
    11. Figure out how to use SillyTavern!

    Benefits of this Method

    • Free
    • Good introduction to running LLMs locally yourself
    • Once you have GPTQ support running, it will open you up to running other LLMs, especially if you get a new graphics cards with more ram, but still not enough to run 13b models fully. More on this in the next section.

    Downsides of this Method

    • You need a relatively new graphics card
    • 4bit quantized models are not as good as their not-4bit counterparts.
    • Installing KoboldAI is pretty simple but can be complicated depending on how tech illiterate you are
    • Open source LLMs like Pygmalion 6B aren’t very good compared to commercial services
    • You’ll wanna spend a bucket of money on a better graphics card just to find out that you essentially already hit the current ceiling of LLM potential on your low end hardware, whoops

    “I just want to experiment with running open source LLM-powered chatbots locally and I have a good graphics card.”

    Do you have a really good graphics card with a lot of VRAM? Like a Geforce RTX 4090 with 24GB of VRAM? Well, you’re in luck, with 16GB of VRAM or more, you can run the full Pygmalion 6B model locally right on your GPU, and it’s pretty easy too. I know I said this last one would be the “hardest” method, but a Geforce RTX 4090 24GB currently costs around $2,000, if we’re counting the new power supply required to power it. So… the hard part is getting the card. But it’s extremely easy to set up Pygmalion 6B on it after you’ve got it installed.

    Using this Method

    1. Install KoboldAI
    2. Once you’ve launched KoboldAI, go to the new UI and hit Load Model. Go to “Chat Models” and pick “Pygmalion 6B”. It’ll download the model and load it up automatically.
    3. Install SillyTavern (runs on macOS or Windows)
    4. When you configure SillyTavern, pick KoboldAI, and put in the URL to your KoboldAI instance (the default should do) and connect.
    5. Figure out how to use SillyTavern!

    Benefits of this Method

    • Free
    • Good intro to running LLMs yourself
    • Extremely easy to get going
    • If you can run Pygmalion 6B entirely in your GPU, you can comfortably share it to the AI Horde and amass kudos that you can use for image generation if you want. More on that in a second.

    Downsides of this Method

    • You’ve installed the non-GPTQ version of KoboldAI here, which means if you want to run something like Pygmalion 13b or Wizard Vicuna 13b locally, you’ll need to go through that dance to run Pygmalion-13b-4bit-128g. So keep that in mind, if you want to run anything past 6b or 7b you’re still going to need to resort to the GPTQ version of Kobold. You’ll also need to learn about splitting these large models into GPU and RAM layers, because 24GB of VRAM is still not enough for them in some cases, but by the time you get to this point of our journey you’ll be so adept at googling for info, you should be able to sort it out yourself.
    • You’ve spent a ton of money on an expensive graphics card but the LLMs you can run locally still mostly suck at this point in time. Thank god it’s useful for gaming, too, huh? And I suppose image generation. And you can share pyg6b to the AI horde for all the other curious people out there checking out this guide and using the AI Horde method, right? How nice of you.

    “That was a lot. Can you just tell me what I should do as if I am unable to make choices for myself?”

    If you just want to have high quality chat or role play with fictional characters, do the first option: SillyTavern + OpenAI. That will send you on a wonderful journey and it will only cost you maybe $15 before you get bored. If you get bored.

    Every other option will yield worse results at the time this is written (2023-05-24). Your desire to do the other options is entirely dependent on external factors. Are you worried that someday corporate overlords will implement stiff content filters against something you enjoy? Then, obviously, downloading Pygmalion 6B and running everything locally can give you some comfort that, lest they wrench your computer from your cold, dead hands, no one is going to take your LLM away from you. It’s also just a fun, nerdy thing to run your LLMs locally.

    But you should know you aren’t currently missing out on some sort of chatbot secret sauce that open source LLMs have that gpt-3.5-turbo does not. The best experience you can get at the moment is paying OpenAI for it. Chatbots powered by gpt-3.5-turbo have better memories, are more creative, stick to a writing style better, write longer responses… all in all, it’s just better. Some day that won’t be true, but that’s not today.

    “What are my other options and anything else should I know?”

    After messing with SillyTavern and KoboldAI for a bit, I looked into other options for running LLMs. Let me tell you what I found. This isn’t a definitive objective opinion on these technologies or products, just my personal opinion and experience with them.

    • I discovered something called “koboldcpp” that can run models without a GPU, and runs a special type of model called GGML. I tried this out so I could use some 13b models locally, and my experience was very poor. I even tried using a GGML version of Pygmalion 6B so I could do a direct comparison, and the results were terrible. It was extremely slow and it did not really work. In my experience, I got essentially gibberish back with no understanding of context. No idea why, but no motivation to figure it out, so I deleted it and I’m sticking with KoboldAI for running LLMs.

    • There’s a lot of talk online of “oogabooga” aka “text-gen-web-ui”. It’s kind of all-in-one KoboldAI and SillyTavern, but I found its Windows setup and configuration to be very confusing compared to KoboldAI and SillyTavern. I managed to get it working eventually, but I found its interface to be clunky and I saw no real reason for me to bother trying to use it. I don’t recommend it, but a lot of people seem to swear by it, so your mileage may vary and more power to you if you like it.

    • There’s an alternative to SillyTavern called Agnaistic. It’s very cool. I used it for a bit and liked it a lot. It’s much slicker in polish than SillyTavern, but it’s not as feature rich in many ways (because it’s brand new and still in alpha). One big benefit, depending on your circumstances, is that it has support for multiple users, so if you have multiple people at home or in your community who want to use chatbots, they can each have an individual account on your Agnaistic instance. You can run it yourself at home just like SillyTavern, but Agnaistic also has a hosted version at https://agnai.chat that you can use connected to your AI Horde API key, or your OpenAI API key, so you can play with it, no local install needed… but I’d be a little wary of handing out my OpenAI API key and putting all my chat history in the hands of a random stranger, but you might not care about that. Without that concern, Agnaistic’s website might truly be the fastest way to just try out a chatbot powered by LLMs on the AI Horde without any real effort.

    • Remember when I talked about how you might want to create a ChatGPT-like helper bot? I built most of a Discord chatbot you can plug an OpenAI API key and a tavern-style character into, to power chatbot on your own Discord server. It’s not fully feature complete yet, but it’s still a fun and functional way to play with a chatbot in a multi-user context with relatively little setup if you’re tech literate. This is an even simpler implementation that doesn’t use any role play nonsense if you’re not into that: https://github.com/NanduWasTaken/gpt-3.5-chat-bot

    • Earlier I mentioned that if you have a fancy graphics card, you can use it to share Pygmalion 6B to the AI Horde. First up, go to https://aihorde.net and see if you can understand what it is. If you can figure that out, register for an API key and store it somewhere safe and permanent (like your password manager). Then you’ll want to figure out where to put that API key in KoboldAI, and name your worker something (like “fartbarf”, why not?). Go back to the Load Model area and toggle the tab that says “Share on Horde”. You should see in the KoboldAI console some stuff indicating pretty quickly that people are using your instance to generate text for their own chatbots. No, you don’t get to see what they are generating (unless you’re a smart hackerman, then obviously you can see everything). What’s cool is that you get Kudos for this that you can then use to use AI image generator interfaces like ArtBot and skip the queue to quickly try out all sorts of different image models.

    • If you get into SillyTavern, seek out SillyTavern-extras. It’s a little complicated to install but, with patience, it adds some nifty stuff, especially the memory extension and sentiment analysis.

    • I mention Pygmalion 6B many times in this guide because it was my introduction point to LLMs. However, there is already a Pygmalion 7B and a Pygmalion 13B that are reportedly much better–but still not on the same level as gpt-3.5-turbo. That said, this tech is advancing so rapidly that it’s totally possible that a month from now, Pygmalion is so advanced that my recommendation to use OpenAI is totally out of date. Just keep this in mind depending how far away from May 2023 you are when you read this.

    “Do you have any general chatbot tips?”

    If you’re trying Pygmalion 6B, you’ll have better luck with it if you truly commit to whatever scenario you are trying to create. Chatbots in general at the moment need a lot of effort to get good content from them. You can’t just message a chatbot saying “Hi” and expect it to craft an interesting interaction out of that for you. I go back to something I read the Google engineer Blake Lemoine say to an interviewer when he was trying to convince people that LaMDA is alive: you have to talk to the chatbot like it’s alive so it will start acting like it’s alive. That rings extremely true based on my interactions with LLMs: the more you treat it like a living breathing person who will pick up on nuance, the more opportunity it will have to genuinely surprise you.

    Having a conversation with a chatbot at the moment is more of a collaboration between you and the LLM, and not just a simple conversation you can engage with passively. You’ll find yourself having to use the “Regenerate” option in SillyTavern’s hamburger menu to give the LLM a mulligan if you got a bad response you’re not happy with. You’ll also find yourself having to simply rewrite parts of the LLM’s response, or the entire response, to keep the conversation on topic or to keep the chatbot from forgetting certain details. You’ll reach moments where you’ve tried to steer the chatbot back on track, and failed, leading to deleting multiple messages at once to try to reset the conversation back to a good state. Without those sorts of efforts, chatbots will start to repeat themselves indefinitely and get stuck into behavioral loops. This can be better or worse depending on the LLM you’re using and its capabilities.

    On top of that, the core “personality” of the LLM can and will influence the way your chatbot behaves in subtle ways. No matter how well crafted your character is, if you do not practice vigilance in how you are interacting with it, it will start to gravitate toward a “average” type of human behavior. The chatbots will also slowly start to adapt to your personality in ways you didn’t intend, too. It’s a really interesting experience to watch happen, but it isn’t magic, it’s actually just a shortcoming of the models at the moment. Something like OpenAI’s gpt-3.5-turbo LLM is not impartial, as even when roleplaying it seems to be prone to more positive behaviors than negative ones. Most LLMs exhibit this characteristic as they are primarily designed to interact with “customers” in a friendly and positive way, not role play as evil demons intent on destroying you and the world around you.

    As time goes on, we’ll get models that are more able to think creatively in a role play context. We’re only at the very, very, very beginning of this journey. If you’re able to be impressed by an LLM at the moment, like I was, I assure you that in a few short years, it’s gonna blow our minds.

    “Wait, how do I use SillyTavern?”

    This one is on you. I found SillyTavern to be really easy to use and figure out. Once you get it connected to an LLM, it’s straight forward. Just watch out for creepy stuff if you start looking for character cards… good luck out there.

  • When I first started my professional development career nearly five years ago now, I was handed a copy of Postico and a business license for it. Since then I’ve used it probably every single working day and a few personal days. It’s a great macOS app, arguably the best, for accessing PostgreSQL databases. Yesterday I saw that there’s a new version: Postico 2.0.

    It fixes an annoyance I barely registered in a massive way: you can now save and categorize self-written SQL queries and it’ll auto-format them for you. Just really awesome and I’m sure this’ll be a super valuable tool in my work flow. I’m just getting settled into a new macOS install as this update comes out, it’ll probably be fun to categorize my existing queries from the old machine into the new one, at some point.

    It has other cool new features too, but I see myself using that one the most.

    ANYWAY… I bought myself a personal license for Postico 2.0, as licenses for the previous version do not carry over. I consider it a permanent part of my development toolkit, so it seems worth the money to me. Check it out!

  • When we moved recently I went on Wirecutter to look at their recommendations for the best Wi-Fi mesh setup. Since my new internet was allegedly gigabit, I bought the ASUS ZenWiFi AX that they recommend.

    It worked great! At first. After a few days, all of a sudden the single node I had wouldn’t use the dedicated 5Ghz backhaul channel: the light on the front of the unit was perpetually yellow. I moved some things around and rebooted everything: the light went white, but then a day or two later the uplink was back to 2.4Ghz mode, hosing the internet speed for anything connected to that node. I tried again to move the units around to make sure they weren’t too obstructed, but nothing would get the 5Ghz backhaul to work consistently.

    I did a little googling, and long story short, I found this Reddit comment, which I’ll copy here for posterity.

    I had the same issues with my XT8. Strong uplink when on 5G-2, but after a while the node would switch to 2.4G where the signal was weak.

    I was able to keep the node on 5G-2 by disabling roaming.

    On the web interface:
    – go to Wireless > Professional
    – select 5Ghz-2 for Band
    – under roaming assistant select “disable”
    – click “apply” and reboot the node

    So I did that and… it worked! Seems like the 5Ghz backhaul connection is very stable now. Figured I would post that here since googling didn’t help until I scrolled all the way to the bottom of a reddit thread, hopefully if you’re having the same issue, this post can help!