A website hosting interactive challenges to test and understand LLM capabilities
About
This project provides a simple and interactive way to test Large Language Models (LLMs) through various challenges. The website is designed to be:
- Easy to understand
- Easy to maintain
- Completely frontend-based (no backend required)
Features
- Interactive challenge selection via dropdown menu
- Adjustable challenge size/complexity
- Various challenge types to test different LLM capabilities
Challenges
For all the challenges we will generate a big chunk of text. For instance a list of numbers or a list of words.
We will also write the questions that will be asked to the LLM.
Finally we also display the answer to the user in a separate box.
All challenges can be generated in different sizes: Small (10 items), Medium (100 items), Large (1000 items), Big (10000 items).
Finding the next number
Same strategy as finding the next word but with numbers.
Finding the next number (not in list)
Same as "Finding the next number" but with a twist, we ask the LLM to find a number that is not in the list.
Finding the next word
We generate a list of words and ask the LLM to find the next word in the sequence.
For this one we generate a list of words, then add a word somewhere in list that is not used somewhere else in the list and we check what is the next word so we have the answer.
Finding the next word (not in list)
Same as "Finding the next word" but with a twist, we ask the LLM to find a word that is not in the list.
Finding missing number (ordered)
Like 1 2 4 5 6, we ask the LLM to find the missing number.
It's 3.
Finding missing number (random)
Like 5 6 3 4 1, we ask the LLM to find the missing number.
It's 2.
Finding value by key (ordered)
1:27
2:153
3:37
4:19
5:61
Finding value by key (random)
3:37
1:27
4:19
2:153
5:61