Two AI robots squaring up to each other

AI Worksheet Wars : Google Gemini Advanced vs. ChatGPT-4

With this week’s release of Gemini Advanced, Google’s latest, premium AI model, we have another platform for language learning content creation.

Google fanfares Gemini as the “most capable AI model” yet, releasing benchmark results that position it as a potential ChatGPT-4 beater. Significantly, Google claims that their new top model even outperforms humans at some language-based benchmarking.

So what do those improvements hold for language learners? I decided to put Gemini Advanced head-to-head with the leader to date, ChatGPT-4, to find out. I used the following prompt on both ChatGPT-4 and Gemini Advanced to create a topic prep style worksheet like those I use before lessons. A target language text, vocab support, and practice questions – perfect topic prep:

Create an original, self-contained French worksheet for students of the language who are around level A2 on the CEFR scale. The topic of the worksheet is “Reality TV in France“.

The worksheet format is as follows:

– An engaging introductory text (400 words) using clear and idiomatic language
– Glossary of 10 key words / phrases from the text (ignore obvious cognates with English) in table format
– Reading comprehension quiz on the text (5 questions)
– Gap-fill exercise recycling the same vocabulary and phrases in a different order (10 questions)
– ‘Talking about it’ section with useful phrases for expressing opinions on the topic
– A model dialogue (10-12 lines) between two people discussing the topic
– A set of thoughtful questions to spark further dialogue on the topic
– An answer key covering all the questions

Ensure the language is native-speaker quality and error-free.

I then laid out the results, with minimal extra formatting, in PDF files (much as I’d use them for my own learning).

Here are the results.

ChatGPT-4

ChatGPT-4, gives solid results, much as expected. I’d been using that platform for my own custom learning content for a while, and it’s both accurate dependable.

The introductory text referenced the real-world topic links very well, albeit a little dry in tone. The glossary was reasonable, although ChatGPT-4 had, as usual, problems leaving out “obvious cognates” as per the prompt instructions. It’s a problem I’ve noticed often, with other LLMs too – workarounds are often necessary to fix these biases.

Likewise, the gap-fill was not “in a different order”, as prompted (and again, exposing a weakness of most LLMs). The questions are in the same order as the glossary entries they refer to!

Looking past those issues – which we could easily correct manually, in any case – the questions were engaging and sensible. Let’s give ChatGPT-4 a solid B!

A French worksheet on Reality TV, created by AI platform ChatGPT-4.

You can download the ChatGPT-4 version of the worksheet from this link.

Gemini Advanced

And onto the challenger! I must admit, I wasn’t expecting to see huge improvements here.

But instantly, I prefer the introductory text. It’s stylistically more interesting; it’s just got the fact that I wanted it to be “engaging”. It’s hard to judge reliably, but I also think it’s closer to a true CEFR A2 language level. Compare it with the encyclopaedia-style ChatGPT-4 version, and it’s more conversational, and certainly more idiomatic.

That attention to idiom is apparent in the glossary, too. There’s far less of that cognate problem here, making for a much more practical vocab list. We have some satisfyingly colloquial phrasal verbs that make me feel that I’m learning something new.

And here’s the clincher: Gemini Advanced aced the randomness test. While the question quality matched ChatGPT-4, the random delivery means the output is usable off the bat. I’m truly impressed by that.

A French worksheet on Reality TV, created by Google's premium AI platform, Gemini Advanced.

You can download the Gemini Advanced version of the worksheet from this link.

Which AI?

After that storming performance by Gemini Advanced, you might expect my answer to be unqualified support for that platform. And, content-wise, I think it did win, hands down. The attention to the nuance of my prompt was something special, and the texts are just more interesting to work with. Big up for creativity.

That said, repeated testing of the prompt did throw up the occasional glitch. Sometimes, it would fail to output the answers, instead showing a cryptic “Answers will follow.” or similar, requiring further prompting. Once or twice, the service went down, too, perhaps a consequence of huge traffic during release week. They’re minor things for the most part, and I expect Google will be busy ironing them out over coming months.

Nonetheless, the signs are hugely promising, and it’s up to ChatGPT-4 now to come back with an even stronger next release. I’ll be playing around with Gemini Advanced a lot in the next few weeks – I really recommend that other language learners and teachers give it a look, too!

If you want to try Google’s Gemini Advanced, there’s a very welcome two-month free trial. Simply head to Gemini to find out more!

An illustration of a cute robot looking at a watch, surrounded by clocks, illustrating AI time-out

Avoiding Time-Out with Longer AI Content

If you’re using AI platforms to create longer language learning content, you’ll have hit the time-out problem at some point.

The issue is that large language models like ChatGPT and Bard use a lot of computing power at scale. To keep things to a sensible minimum, output limits are in place. And although they’re often generous, even on free platforms, they can fall short for many kinds of language learning content.

Multi-part worksheets and graded reader style stories are a case in point. They can stretch to several pages of print, far beyond most platform cut-offs. Some platforms (Microsoft Copilot, for instance) will just stop mid-sentence before a task is complete. Others may display a generation error. Very few will happily continue generating a lengthy text to the end.

You can get round it in many cases by simply stating “continue“. But that’s frustrating at best. And at worst, it doesn’t work at all; it may ignore the last cut-off sentence, or lose its thread entirely. I’ve had times when a quirky Bing insists it’s finished, and refuses, like a surly tot, to pick up where it left off.

Avoiding Time-Out with Sectioning

Fortunately, there’s a pretty easy fix. Simply specify in your prompt that the output should be section by section. For example, take this prompt, reproducing the popular graded reader style of language learning text but without the length limits:

You are a language tutor and content creator, who writes completely original and exciting graded reader stories for learners of all levels. Your stories are expertly crafted to include high-frequency vocabulary and structures that the learner can incorporate into their own repertoire.

As the stories can be quite long, you output them one chapter at a time, prompting me to continue with the next chapter each time. Each 500-word chapter is followed by a short glossary of key vocabulary, and a short comprehension quiz. Each story should have five or six chapters, and have a well-rounded conclusion. The stories should include plenty of dialogue as well as prose, to model spoken language.

With that in mind, write me a story for French beginner learners (A1 on the CEFR scale) set in a dystopian future.

By sectioning, you avoid time-out. Now, you can produce some really substantial learning texts without having to prod and poke your AI to distraction!

There may even be an added benefit. I’ve noticed that the quality of texts output by section may even be slightly higher than with all-at-once content. Perhaps this is connected to recent findings that instructing AI to thing step by step, and break things down, improves results.

If there is a downside, it’s simply that sectioned output with take up more conversational turns. Instead of one reply ‘turn’, you’re getting lots of them. This eats into your per-conversation or per-hour allocation on ChatGPT Plus and Bing, for example. But the quality boost is worth it, I think.

Has the section by section trick improved your language learning content? Let us know your experiences in the comments!

Does AI have a noun problem? Strategies for avoiding it.

AI Has A Noun Problem : Let’s Fix It!

If you’re using AI for language learning content creation, you might have already spotted AI’s embarrassing secret. It has a noun problem.

Large Language Models like ChatGPT and Bard are generally great for creating systematic learning content. They’re efficient brainstormers, and can churn out lists and texts like there’s no tomorrow. One use case I’ve found particularly helpful is the creation of vocab lists – all the more so since it can spool them off in formats to suit learning tools like Anki.

But the more I’ve used it, the more it’s become apparent. AI has a blind spot that makes these straight-out-the-box vanilla lists much less useful than they could be.

A fixation with nouns.

Test it yourself; ask your platform of choice simply to generate a set of vocab items on a topic. Chances are there’ll be precious few items that aren’t nouns. And in my experience, more often than not, lists are composed entirely of noun items and nothing else.

ChatGPT-4 giving a list of French vocabulary items - all nouns.

ChatGPT-4 giving a list of French vocabulary items – all nouns.

It’s a curious bias, but I think it has something to do with how the LLM conceives key words. The term is somehow conflated with all the things to do with a topic. And nouns, we’re taught at school, are thing words.

Getting Over Your Noun Problem

Fortunately, there’s therapy for your AI to overcome its noun problem. And like most AI refining strategies, it just boils down to clearer prompting.

Here are some tips to ensure more parts-of-speech variety in your AI language learning content:

  1. Explicit Instruction: When requesting vocabulary lists, spell out what you want. Specify a mix of word types – nouns, verbs, adjectives, adverbs, etc. to nudge the AI towards a more balanced selection. When it doesn’t comply, just tell it so! More verbs, please is good start.
  2. Increase the Word Count: Simply widening the net can work, if you’re willing to manually tweak the list afterwards. Increase you vocab lists to 20 or 30 items, and the chances of the odd verb or adjective appearing are greater.
  3. Contextual Requests: Instead of asking for lists, ask the AI to provide sentences or paragraphs where different parts of speech are used in context. This not only gives you a broader range of word types, but also shows them in action.
  4. Ask for Sentence Frames: Instead of single items, ask for sentence frames (or templates) that you can swap words in an out of. For instance, request a model sentence with a missing verb, along with 10 verbs that could fill that spot. “I ____ bread” might be a simple one for the topic food.
  5. Challenge the AI: Regularly challenge the AI with tasks that require a more nuanced understanding of language – like creating stories, dialogues, or descriptive paragraphs. This can push its boundaries and improve its output.

Example Prompts

Bearing those tips in mind, try these prompts for size. They should produce a much less noun-heavy set of vocab for your learning pleasure:

Create a vocabulary list of 20 French words on the topic “Food and Drink”. Make sure to include a good spread of nouns, verbs, adjectives and adverbs. For each one, illustrate the word in use with a useful sentence of about level A2 on the CEFR scale.
Give me a set of 5 French ‘sentence frames’ for learning and practising vocabulary on the topic “Summer Holidays”. Each frame should have a missing gap, along with five examples of French words that could fit in it.
Write me a short French text of around level A2 on the CEFR scale on the topic “Finding a Job in Paris”. Then, list the main content words from the text in a glossary below in table format.

Have you produced some useful lists with this technique? Let us know in the comments!

Icelandic Noun Master - an app with an appreciative audience.

Do It For An Audience

It’s nice to be appreciated. And sometimes, an appreciative audience can be just the boost you need to get back into gear.

I received some lovely feedback this week about an app I’d almost completely forgotten about. It all related to a very active Icelandic phase I was going through a couple of years back. At the time, I was enjoying a particularly fierce battle with noun declensions, but suffering from a dearth of resources to help (fellow Icelandic learners will relate).

There’s a good piece of advice in this situation. If there’s no help forthcoming, help yourself.

To get a handle on those noun tables, I put together a quick ‘n’ simple app to drill those declensions. I used Java and Android Studio (it’s my job, after all), but there was no prerequisite level of tech – it’s something that could just as easily take life in a site like Quizlet or Educandy.

The idea was basic: a set of multiple choice activities to drill Icelandic noun endings, separately by gender, or altogether. It just needed a bit of time to put together questions and prompts from the grammar guides I had available to me. And the result? A really effective five-minutes-a-day app for getting those endings into memory.

The added benefit of putting it together as a mobile app was that it was ready-bundled to share on to others. I released it as Icelandic Noun Master on Google Play as a free app, and watched the downloads slowly clock up. It’s still there, quietly helping anyone who needs it.

Learning by Making

DIY resource production – for yourself and for others – is a language learning strategy that can yield surprisingly positive results. For a start, resource creation gets you thinking deeply about your learning material, and how to transform it into a clearer, easily testable format. To make questions from it, you have to step away, look at it from a different angle, turn it inside out, think about it in ways that perhaps weren’t obvious on first glance. It’s like turning a jigsaw puzzle upside-down for a fresh perspective, and suddenly spotting where a piece goes. That see it in a different way benefit, incidentally, is why teaching to learn is likewise such a good strategy.

But there’s another intended side-effect, an almost hypnotically effective one. In the creation of resources, you can drift into an almost automaton-style collating of material, sourcing and listing sample sentences, questions or tabular data. It’s a kind of flow state that encourages foreign language material to bed itself in almost by a process of osmosis. Even if it doesn’t quite become active knowledge in one fell swoop, it lays the ground for it to become so later.

Keep ’em Coming

So, in these ways (and probably many more), an appreciative audience can be a useful tool for a language learner. And of course, there’s also that feeling that what you’re doing has impact and usefulness – and that can work wonders for your motivation. In any case, it’s got me thinking that there’s a bit of life left in the trusty old Icelandic Noun Master yet. I’ll be returning to it now, to spruce it up, and revise my own Icelandic. And maybe I’ll even add an iOS version to the mix, too.

Have to keep that audience happy!

Programming in binary code

Love languages? Try programming!

Programming languages have a lot in common with human languages. For a start, they all have a very particular vocabulary and syntax. You need to learn the rules to assemble meaning. And both machine and human languages are tools for of turning concepts in our heads into action in the real world.

My love of languages blossomed around the same time as my fascination with computers. I’d tinker around in BASIC on my Commodore VIC-20 as a little kid, getting that early PC to just do things. (I know, that really dates me!) And today, I’m lucky enough to have made a career combining those two strands together as an educational software developer.

Works in progress

That said, it’s a career that never stands still. And, just as with human languages, it’s important to maintain and improve your skills all the time. In the same way that ‘fluency’ is an ill-defined and unhelpful ‘completion’ goal, you never really stop learning in the tech industry. There’s no end-point where you down tools, show your certificate, and say “I know it all now!“.

A fantastic source of development training for me of late has been the peer-tutorial site Udemy. I like the nature of the platform, allowing ordinary folk the chance to share their skills (and earn a bit of money from it, too). I also like the pick-and-choose nature of it, where you pay per course, rather than an all-in subscription. That’s one reason I always felt I wasn’t getting enough usage from the industry training giant, Lynda.com.

In fact the only downside to Udemy is its odd pricing model. Courses are listed under a ‘normal’, inflated price, but are almost always available at a discount. This discount varies, meaning that users end up course-watching until the price is lowered. Then they pounce, usually at a very reasonable rate of around £10 or so. I realise that the commercial psychology behind it is to increase the sense of bargain, but it does seem a little convoluted.

What I’m working on

In any case – there are some gems of courses on there. That goes especially for those who fancy learning some programming for educational applications. For a brief overview, here are some of the fantastic resources I’ve found useful:

Swift 4 and iOS

Apple introduced the Swift language as a successor to the clunky Objective-C language in recent years. It’s much easier to learn, in my opinion, and is more cross-skill compatible with other programming languages. Instructors have embraced the new language on Udemy, and amongst the best courses are the ones from tutorial guru Ray Wenderlich, and London-based developer Angela Yu. I intended to use their courses as refreshers, but have learnt a huge amount from both of them.

Android and Kotlin

Kotlin has a similar story to Swift, as a new language positioned to supersede and older one. That old one is Java, which is arguably a lot more useful and widespread than Objective-C. However, Kotlin is remarkably similar to Swift in syntax and usage. As such, it’s a pretty good choice to add to your collection if you are aiming for both iOS and Android development.

There is an old-school Android developer on Udemy, Tim Buchalka, who really knows his stuff. He’s my go-to for all my Android courses, and his Kotlin course is probably the most accessible and practical out there.

Not all hard work!

It’s not all hard work, of course. I take a couple of courses just out of interest or curiosity. As a programmer, I’ve always felt a little inferior about my design and illustration skills. Not only that, but I’m often a little jealous of how in the zone and mindful digital artists can get when working. To that end, I’ve been following a great course on creating digital art on the iPad with the Procreate app. Because not everything has to be about languages, programming or otherwise!

 

Describe It! Speaking drill game for fun practice prompts

I’m always looking new ways to make speaking practice fun. It was BBC’s Just A Minute that inspired me to put this basic drill activity together. From a bank of many random concepts – TV shows, celebrities, countries, landmarks – the program draws one each turn. You then have sixty seconds to describe and discuss it without pausing.

Describe It! Speaking dill game

Describe It! Speaking dill game

It’s perfect for adding into your pre-lesson warm-up routine. And you can tailor it to your own level and needs – simply make your descriptions / spontaneous monologues as simple or complex as you can handle. Try answering these questions about the topics that pop up if you’re stuck for words:

  • What is it?
  • What do you think about it? Do you like it?
  • Who does it involve?
  • What else is it connected to? And is it controversial in any way?

Click here to open the prompt applet in a new window. As an HTML5 widget, it should run across all sorts of platforms.

Help it grow!

I put this together originally for my own use, so some of the concepts might seem a bit UK-centric. However, if you have some good ideas for items to add to the data bank, please share them in the comments or tweet me! I’ll add good ones to the activity on an ongoing basis. I hope others find it useful (and appreciate the silly humour that drives it! 😄).

Eurovision 2017 Logo

Add some Eurovision sparkle to your language learning!

The Eurovision Song Contest may be over for 2017 (congratulations, first-time winner Portugal!), but it can still be a sparkling, magical resource for teaching and learning modern foreign languages.

Eurovision and languages have gone hand-in-hand for me since my early days of crazy fandom. Aged 15, I became intrigued by this exotic musical competition full of unusual-sounding tongues. It fuelled my nascent passion for languages, and it’s a dual obsession that continues to this day. Eurovision is why I can say ‘love’ in 20+ languages. It’s why I know all the country names so well in French. And even with the explosion of English-language songs since 1999, it can be a wonderful learning resource for ‘normal’ folk, too! 

Here, I’ve collected a few ideas for getting started with Eurovision as a language-learning resource. Admittedly, the links here will be old-hat to dyed-in-the-wool fans like me. But if you’re just a marginally less insane lover / learner / teacher of languages, you might find something useful in here for your own learning.

Eurovision can be fun, serious, silly, touching – but most of all, memorable. And it’s that memorability that gives the material salience and staying power when you’re learning a language!

Videos and lyrics

As talking points for a lesson, Eurovision clips are perfect. They’re short – the three-minute rule makes sure of that – and they are wonderful time capsules of fashion, too, giving you loads of material for discussion. Do you like the stage / set? What do you think of the clothes? Would that song be a hit today? You can go on and on.

The official YouTube channel of the Eurovision Song Contest is the first stop for video clips of songs from past contests. If you can’t find the exact entries you want there, a quick search on YouTube along the lines of “Eurovision YEAR COUNTRY” (like “Eurovision 2017 France”) will always throw up some good results.

Waxing lyrical

For a bit of text support, there is a fantastic lyrics site with every Eurovision entry to date on it: The Diggiloo Thrush (you may have to stop tittering at the name before you look it up).

I’ve used Eurovision lyrics to mine for fresh vocab. For instance, I’ll take a song I like in a language I’m learning, look up the text, and note any new words in my vocab bank (I use Anki currently for this). If I really love a song, I’ll also try to learn it, so I can sing it in the privacy of my own shower. T.M.I., I know, but whatever it takes to learn!

Eurovision gapfills

If you’re teaching others, you can use lyrics to make interactive activities for your students, too. Copy and paste your chosen song text into a document / Textivate game or similar, removing some of the words to make a gapfill. Play the song to the students and get them to fill in the gaps as they hear them. It’s a brilliant way to focus the ears on the sounds of the target language.

There are lots of ways to approach this with different objectives. For instance, you could remove all the non-content words, like ‘and’, ‘but’, ‘then’ and so on. That hones the attention on all those little connective words that we need to make our language flow. Alternatively, take out the content words (you’ll find ‘love’ quite a lot in Eurovision songs!) to practise concrete, topical vocab.

Language awareness

A game I liked to play with my own language classes, back in the day, was ‘guess the language’. I’d prepare clips of Eurovision songs in a range of languages including the one(s) the class was learning. Of course, you can throw in some sneaky difficult ones. Dutch is great, if they’re learning German, or Italian if they’re learning Spanish, to throw them off the scent.

It’s an engaging and competitive way to get students thinking about how languages are related to one another, and where the language they’re learning fits in to the bigger picture. It’s ‘meta-knowledge’ in the sense that it’s about what they’re learning more generally – language – than knowledge of the language itself. But it’s an excellent way to show the target language within its global context.

Eurovision: national reactions

National press can go crazy over Eurovision, generating a raft of headlines and articles for consumption. Right after a contest, you can easily find web articles from countries that did either well or badly, by simply going to the homepage of the national broadcaster. This article from Norwegian broadcaster NRK, for example, describes the high mood of the team after scoring a top ten placing in Kyiv this year.

Why are these articles useful? Well, they’re usually quite simple to read. They’re about a well-known, universal field – music and entertainment – so they won’t contain too many complex notions like other news articles might. Also, they’re full of those vocab items like dates, numbers and such like, which are simple, but a pain to learn. Excellent practice!

Where to find broadcaster links? Well, Wikipedia provides a very handy list of EBU member stations at this link. Also handy for looking up programming in your target language, even when Eurovision isn’t on!

Eurovision is a marvellous, fun, colourful, diverse and happy medium for language learning. What’s more, all of the material is freely available online for you to get creative with. With over 60 years of history, there’s a treasure of resources to play with, so get out there and bring some Eurovision magic into your language learning!

dictionary

Getting lost in languages: finding your flow

How often do we hear others dismiss language learning as “too hard” to bother?

In my own long and varied experience with MFL, it’s a charge I’ve heard frequently levelled at languages, as much from frustrated students as from family and friends. “I’d love to learn a language, but I’m just no good at it” is such a common defence; “I’ve got a terrible memory for languages” is another.

But what if expending too much effort is part of the problem? This isn’t to say that there’s some magic, easy method to acquire a working knowledge of a language in a short amount of time. No subliminal headphones-while-you-sleep shortcuts, I’m afraid.

Rather, we should be challenging the over-serious, head-breaking, traditional model of language learning; that slightly authoritarian, sit-down-and-learn-your-grammar reputation that MFL has (rightly or wrongly) earnt over the decades.

Recently I’ve been working on interactive resources in Maori and Latvian, two languages I know next to nothing about. A lot of the groundwork for this involved pretty repetitive copy-pasting to create resource files for apps. However, despite the fairly automatic nature of the task, I found myself noticing and picking up language patterns almost subconsciously  during the process. After more than 100 Latvian verb conjugations, for example, you start to recognise present tense endings like -u/-i/-a/-m/-t/-a and other groups more or less instinctively.

In the zone

Not only that, but turning into a bit of a copy-paste automaton for an hour or so was an easy – even relaxing – experience. Talking about it with a  colleague, I likened it to ‘taking a stroll’ through the language. I’d entered that mindful state of ‘being in the zone’, or flow, as described by positive psychologists like Csíkszentmihályi. I had, in effect, created the perfect mind conditions to enjoy and absorb working within the foreign language, almost without any conscious effort.

I do this kind of task very often in my line of work, unsurprisingly, it’s led me to a head chock full of vocab and grammar snippets that I never really intended to learn, but somehow, fortuitously, did anyway. It leads me to re-evaluate the kinds of learning task that we often dismiss in MFL, those that seem to have little worth on the surface, like word searches and simple matching activities. I’ve often a guilty ‘word search snob’ myself, but it’s likely worth rethinking their poor reputation amongst MFL educators in this light. Food for thought when considering whether to include such ‘low-level’ tasks in your language learning regime or resources!

If they’re engaging enough to spark a little of that flow, contain a fair amount of language patterns and paradigms with clear meaning,  then maybe, just maybe, ‘grunt’ language learning tasks have a valuable place in learning.