Ghigliottina Solver

Context

"La Ghigliottina" is the final game of the Italian TV show L'Eredità: five clues, unrelated to each other, each linked to one hidden word. The link is almost always lexical or cultural — the solution forms a compound, an idiom, or a frequent collocation with each clue (doppio, carta, soldi, pasta, regalo → pacco). The player has 60 seconds.

The Natural Language Processing exam required a system that solves the game within the time limit and generates a description explaining the link with each clue. Dataset: 965 training games, 100 test games — and only 22% of test solutions appear in training. Memorisation is not enough: genuine association reasoning is required.

Core idea

The finding that drives the whole project is empirical: general-purpose LLMs are weak at solving this game (GPT-4 reaches ~4% in the literature), whereas a knowledge-based ranker grounded in word associations — the historical approach to this game (OTTHO, UNIOR4NLP) — is far more reliable and essentially instantaneous.

So: division of labour. A knowledge-based solver finds the solution; the LLM only generates the description — where generation is its actual strength.

How it works

5 clues ─► [A] candidate generation ─► [B] coverage-based scoring ─► SOLUTION
                                                                  └─► description (LLM)

(A) Association knowledge base. Three sources, all indexed by co-occurrence:

1. Multiword expressions and proverbs provided by the course (~36,000 entries, including the De Mauro dictionary)
2. Multiword Italian Wikipedia titles treated as expressions ("Conquista del West" links conquista and west)
3. Collocations mined from the Paisà corpus — for each clue, the words co-occurring within ±3 tokens with positive PMI

Given the five clues, the candidates are all words associated with at least one of them.

(B) Coverage-based scoring. Each candidate is scored as score = W · coverage + mwe_strength / √frequency. Coverage — how many of the five clues the candidate is linked to — dominates: a word linked to all five beats one linked to two, which is exactly the game's logic. Dividing by √frequency is an IDF-like penalty that stops common words (dire, grande) from polluting the ranking. Hyper-parameters were chosen on a 200-game dev set, leaving the test set untouched until the final evaluation.

(C) Description. Generated by Qwen2.5-7B running locally (via Ollama), with two few-shot examples from the training descriptions, in the gold style. Local because it is free, unlimited, and not subject to the 60-second limit — which applies to solving only.

Results

• 47% top-1 accuracy on the solution, MRR 0.55, gold in the top-50 for 82% of games
• 0.01 s/game — zero games over 60 seconds
• Description: BERTScore-F1 0.73 (multilingual BERT), ROUGE-1 0.33, BLEU 6.73
• Progression of design choices: 21% (raw MWE count) → 25% (coverage + rarity) → 36% (Wikipedia titles, recall 63%→89%) → 47% (corpus collocations)

What was tried and dropped

The ablation is half the value of the project:

• fastText embeddings — helpful for recall before the Wikipedia titles (25%→31%), redundant and slightly harmful afterwards → out of the final system
• Graph scorer (Personalized PageRank) — spreading activation from the five clues does not beat the direct ranker: the bottleneck is data coverage, not the algorithm
• LLM solution selection — degrades the ranker: local models score ~0% on a diagnostic test, the 14B exceeds 60 seconds. Confirms that LLMs do not help to solve this game

Takeaways

The decisive lever wasn't the algorithm but the breadth of the knowledge base: every accuracy jump came from adding a source of associations, not from refining the scoring. And the negative result is worth as much as the positive one — knowing where an LLM doesn't help is a design skill, not a defeat. The pipeline is conceptually RAG taken to the extreme: retrieval decides the answer, the model merely explains it.

Individual project for the NLP course at Università degli Studi di Bari Aldo Moro.