Tag: LLM

Ethical AI : Give Your Local Models a Window on the World

Leave a comment

AI has become one of the most divisive and controversial emerging tech developments of our age. And not without reason – its impacts on the environment and on society are hotly debated and still not completely understood.

It’s one of the reasons that ethical AI is such a promising field right now. Using free tools on any medium-specced machine, you can beef up your workflows without burdening data centres in exchange for your privacy.

Perhaps you too are already using local LLMs as a more environmentally responsible content aid. They’re a great choice for assisting in activity creation in “mainstream” languages; the latest small models can comfortably match the output of the online beasts like Claude and ChatGPT.

But you may have already run up against the “black box” limitation. Out of the box, locally run LLMs have only their own resources, and no connection to the outside world for retrieving up-to-date info or fact-checking.

Tooling AI

There is a solution – and if you’re running models in LM Studio, it’s pretty easy to implement. A number of tools, or plug-ins, are available to extend the capabilities of your models. They range from simple everyday functions like a current date check, to internet searching and even file system access. When browsing models in LM Studio, tags indicate which are ready for tool use – which, in practice, is nearly all recent releases.

One word of caution – it’s important to take some care in selecting and installing tools. Community users are generally the tool developers, rather than the LLM providers. As such, you should always do due diligence and check sources before installing them. Building these assumptions into practice has been a baptism of fire for users at the bleeding edge of AI – you may well have come across recent stories about insecurities in certain third-party tools for the ‘agentic’ system, OpenClaw.

That said, LM Studio hosts a library of third-party tools on its website that should have passed community checks. These include a Wikipedia search, which instantly gives your queries a fact-checking foundation (allowing for the occasional Wikipedia factual error or two!). If that’s not enough, then there’s that full web search tool too. To use this, you need to set up a search provider – the free and ethical SearXNG is recommended, and it’s worth the fiddling around to set it up. (Again, exercise plenty of care when allowing your local models to consume external data.)

And back to that prosaic old date check – it’s actually more useful than you might at first think. As LLMs are trained on static text data, they have a memory cut-off. There’s generally an ‘awareness’ of this (insofar as we can call it that). Try, for instance, asking a local model what was in the news yesterday. It might well respond by stating its knowledge cut-off date (if it doesn’t make something up). Install a date checker and you add a corrective to this behaviour. That web search is much more effective if a model ‘knows’ when now is.

New tools are being developed all the time, bringing local model use even closer in line with online AI portals. For many, that’s a big step closer to becoming a viable, ethical replacement for them. Explore, experiment and share – I’d love to know what other tools you’ve found useful in the comments!

Shelves of helpful robots - a bit like Poe, really!

Which LLM? Poe offers them all (and some!)

Leave a comment

One of the most frequent questions when I’ve given AI training to language professionals is “which is your favourite platform?”. It’s a tricky one to answer, not least because we’re currently in the middle of the AI Wars – new, competing models are coming out all the time, and my personal choice of LLM changes with each new release.

That said, I’m a late and recent convert to Poe – an app that gives you them all in one place. The real clincher is the inclusion of brand new models, before they’re widely available elsewhere.

To illustrate just how handy it is, just a couple of weeks ago, Meta dropped Llama 3.1 – the first of their models to really challenge the frontrunners. However, unless you have a computer powerful enough to run it locally, or access to Meta AI (US-only right now), you’ll be waiting a while to try it.

Enter Poe. Within a couple of days, all flavours of Llama 3.1 were available. And the best thing? You can interact with most of them for nothing.

The Poe Currency

Poe works on a currency of Compute Points, which are used to pay for messages to the model. More powerful models guzzle through compute points at a higher rate, and models tend to become cheaper as they get older. Meta’s Llama-3.1-405B-T, for example, costs 335 points per message, while OpenAI’s ChatGPT-4o-Mini comes in at a bargain 15 points for each request.

Users of Poe’s free tier get a pretty generous 3000 Compute Points every day. That’s enough credit to work quite extensively on some of the older models without much limitation at all. But it’s also enough to get some really useful (8-ish-requests daily) use from Llama 3.1. And, thanks to that, I can tell you – Llama 3.1 is great at creating language learning resources!

Saying that, with the right prompt, most of the higher-end models are, these days. Claude-3.5-Sonnet is another favourite – check out my interactive worksheet experiments with it here. And yes, Claude-3.5-Sonnet is available on Poe, at a cost of 200 points per message (and that’s already dropped from its initial cost some weeks back!). Even the image generation model Flux has made its way onto the platform, just days after the hype. And it’s a lot better with text-in-image (handy if you’re creating illustrated language materials).

Poe pulls together all sorts of cloud providers in a marketplace-style setup to offer the latest bots, and it’s a model that works. The latest and greatest will always burn through your stash of Computer Points faster, but there’s still no easier way to be amongst the first to try a new LLM!