I know everyone is hyped about 3.8, but honestly, I’m still polishing 3.6. Plus, I hate the way 3.8 talks, so…
I keep seeing tons of posts debating what open source is actually good for, so I wanted to share my own take with this post.
Qwen 3.5 27B was my very first agent, and my journey kicked off around May 31st. Over the past 4+ months, I’ve learned a ton and wanted to pay it forward to anyone just getting started. There’s clearly a huge wave of new users right now looking for guidance, but people are often hesitant to ask, or simply don’t know what to ask. So, here are 10 empowering things you should know!
(Oh, and by the way, I highly recommend llama.cpp. That’s what I used to pull off all these tricks.)
1. Build neat little apps for yourself
Qwen is remarkably good at coding apps for all kinds of use cases. In fact, I’m writing this very post in a custom Notepad replacement app I built myself—complete with full text-processing capabilities, a sidebar, light/dark mode, and more (see image below). Whenever you run into a problem, your first thought should be: “Can I just write an app to make this easier?” The answer is usually yes.
2. Craft your own Chat UI frontend
If you don’t feel creative enough, just copy an existing one. LLMs are phenomenal imitators. I like Claude, so I just cloned its interface into my own custom frontend and tweaked it to fit my needs:
3. Build a ReAct loop
This is the mechanism that allows your model to call tools, inspect the results, and call them again (repeating as many times as necessary). Qwen can write these in its sleep, and in plenty of different ways. It can spin up loops wherever you need them, and if you’re building a custom Chat UI frontend, you’re definitely going to need this.
4. Write MCP tools
This is my absolute favorite. It can write virtually any kind of MCP (Model Context Protocol) tool you need. You don’t have to rely on third-party tools that might pose security risks. Just tell it you need an MCP tool for a specific task, and it will code it up. It may or may not require a standalone MCP server, but even if it does, Qwen can handle the server-side code without breaking a sweat.
5. Spawn sub-agents
This is another super simple task that Qwen handles effortlessly. Just have it create a sub-agent generator (which is basically just a Python file). Then, whenever you need to spawn a sub-agent, you can either run the script or have it generate an MCP tool to spin them up on demand. It’s that easy. Oh, here’s a neat pro-tip: MCP tools chew through a lot of context, so a clever workaround is to spawn a sub-agent with its own isolated context limit, then have the main agent delegate tasks to it. Basically, you can double, triple, or further multiply your effective context limit this way. And yes, you can maintain context across multiple turns, though you’ll need to build a pipeline runner for that (see below).
6. Build a pipeline runner
Ever heard of n8n? At its core, it’s a pipeline runner. It’s packed with features, but it has some limitations and quirks—things I wished it could do, but couldn’t. Then I realized building a similar tool with Qwen was a breeze. If you want a frontend UI, it does take some design effort, but it’s totally doable. I designed a custom UI myself, but after tweaking it for a while I got tired of tinkering, so I ended up just cloning n8n’s frontend and customizing it for my setup, haha. Another pro-tip: You can chain multiple agents sequentially in a pipeline runner, but if you want each agent to maintain its own independent context, make sure you enable cache-idle-slots in llama.cpp. This swaps out the KV cache on each turn and offloads the previous one into system memory (CPU RAM). There are a few subtle details to this, but Qwen can walk you through the whole setup—just ask it.
After experimenting with multiple custom UIs, I finally decided to clone the look and feel of n8n and build my own version.
7. Build a model launcher (with model/Jinja template selection in llama.cpp)
This is my daily driver. It’s a launcher that lets me select any model in my directory and configure its parameters. Once I dial in settings I like, I save them under an alias. Next time I launch, I just pick that alias, and it spins up the model with those exact parameters. That means I can have 10 different parameter presets for the exact same model if I want—all tied to unique aliases. Pro-tip: I recommend adding your Jinja templates to the launcher too, allowing you to select tailored templates for different model architectures.
I created custom Jinja templates for different model types.
8. Install, test, fork, and rebuild open-source apps from the web
The reason I switched to llama.cpp was because LM Studio’s inference felt too slow, but I didn’t want the headache of figuring out the entire manual install process. So, I just had my agent download and install llama.cpp for me. I’ve been using it ever since. It can also tweak builds, fork GitHub repositories, and more. For example, while I was busy working on another project, I heard about a couple of new text-to-speech (TTS) apps, so I had my agent pull and install Moss and OmniVoice, generate a few audio clips to verify everything was working, and ping me when done. Sure enough, everything ran smoothly. It’s incredible for multitasking.
9. Give your agent temporal awareness of the outside world
Ever used a cloud model that literally argues with you about today’s date? Or wished your model could tell when it’s nearing its maximum context window so it can proactively start summarizing the session? What you can do is inject metadata into your agent at session start and during every single turn (ideally both). Feed it the current date, time, used context length, max context window, software/hardware environment—basically whatever you want.
Once you have this, a whole world of tricks opens up. For instance (maybe this should’ve been its own bullet point under “another thing you can do with Qwen”, but whatever), I set up my agent to automatically hot-swap the underlying model once it hits the context threshold:
I can start a session with a fallback model configured. Because my agent always sees its real-time context usage, once usage spikes too high, it seamlessly switches over to a fallback model with a larger context window—all without restarting the server.
10. Equip your agent with instant boot-up memory
Whenever a session wraps up, my agent writes a concise summary of what we discussed. You can configure your startup script or launcher to stitch various data sources into its context upon boot. For instance, my setup always loads the most recent session summary alongside the previous day’s summary on startup, giving the agent a comprehensive snapshot of recent events. You can extend this with any extra docs you want—let your imagination run wild. For example, my agent also pulls in reminders and its own reflections from the end of the last session (though I no longer recommend keeping reflections, as I’ve realized they tend to pollute context—more on that in a future post).
Bonus: (I remembered this right as I was finishing up, and I felt I had to mention it, because frankly it’s one of the most critical takeaways)—Build your own test/evaluation harness. Everyone is talking about Pi, Hermes, and the like. I decided to just use Qwen to build my own framework from scratch. It takes more work, but the payoff is vastly higher because you truly understand how these systems operate under the hood, and you can leverage that knowledge to craft uniquely tailored solutions. For example, the moment I learned how to use a proxy, I figured out how to unlock all sorts of awesome features using context compression, history modification, data sanitation, Jinja template wizardry, hooks, and more.
Anyway, I hope this sparks some ideas for newcomers. I really recommend learning from the ground up—your skills will compound much faster that way. Get your Qwen hooked up to web tools as soon as possible so it can do research and help you along the way.
I’ve discovered tons of other tricks that I might save for future posts, but if you can get the things above up and running, you’re already on the right track. Cheers!
Grunge
P.S. I come from the old-school forum and DeviantArt days, where heavy use of Ctrl+B/I/U and Shift+L for formatting was second nature. I’ve been writing heavily bolded posts like this for over 20 years. 100% written by hand. Receipts are attached at the bottom of the post if anyone cares. Adding this disclaimer solely because nicely structured formatting always attracts trolls screaming “this is AI-generated slop.”
Top Comments
suprjami (⬆89): Adding this disclaimer solely because nicely structured formatting always attracts trolls screaming “this is AI-generated slop.” GrungeWerX (OP) (⬆22): I feel you. That’s why I usually toss in a disclaimer whenever it crosses my mind.