Generated puzzles

Generated Puzzles

What is particularly timeful about this article is that someone recently posted in my server, and thus it became a topic about 'generated puzzles' there coincidentally:

IQuick 143:

Generating puzzles automatically is quite difficult, because to make puzzles that are interesting, you need to hit these interesting moments, which has two obstacles:
1) Your random distribution on the space of puzzles needs to hit puzzles that are coherent enough often enough to be useful.
2) You need to somehow be able to tell when a puzzle is good without having the "I know it when I see it" of a human designer.

Putting just random objects together is likely to hit a roadblock with 1)
Putting together known constructions is likely to hit a roadblock with 2) (because if you learn the patterns the builder uses to construct the levels, there's usually not much more to find)

One approach I found that has limited success is when you can create an ordering relation on your levels that orders them by inclusion of the set of solutions. (An "easier" level accepts all solutions of a "harder" level and possibly more, like a sokoban level with no walls vs same sokoban level without walls)

You can then take some level (possibly even random jumble of things) and do random things to make it "easier" until it's solvable, and then do other things to make it "harder" until you make it unsolvable and repeat this process as a kind of random walk. This sometimes produces interesting levels because there's something that would block solvability if the level was "harder".

Of course I do not agree with this, the problem of 'puzzles that interest people that are generated' have already been solved by many games. 14 Minesweeper Variants is probably the most currently well known one, but its SAT solversemiexp/csugarA C++ reimplementation of the Sugar CSP solver, which turns constraint problems into SAT. Has Python bindings.github.com is too complex to explain here. So here instead I've set up a very simple Minesweeper creator algorithm as an example instead. But note that all of these generated puzzles are 'bruteforcing' in a sense, and there are 'work backwards' methods as well as 'work forwards' methods.

Solve to unlock the slideshow of how it was generated. Prev/Next generator switches between the Minesweeper one and the area numbers one.

(also play MinescryerMinescryerAn unconventional Minesweeper variant created for the Confounding Calendar 2023 that subverts traditional Minesweeper rules. In a way its rule discovery is literal.icely.itch.io and Arrows PuzzleIcely's Arrows PuzzleA five-page paper puzzle book about arrows.icely.itch.io which have very high quality reveals)

Oh by the way the inventor of the quote of this article has responded in certain ways in a good way, mentioned that there is also a distinguishment between games with more open ended solutions where creating something like this is harder.

Remaining AI-Related Part of Article

An oddly common sentiment in ~2024-2026 ever since LLM's, is "you didn't bother to write it so why should I read it" or "why would I play something a human didn't make". This makes a lot of sense for low-effort trash that's been polished to look better by LLM's, and especially because LLM's as of now still are hugely lacking in game design sense and taste.
Obviously there is something generally true with that and to see "AI, bad, skipped" is very sensible, but also because there's a lot of "human slop" putting out LLM work as if it is geniusThe sad part is that it's probably TRUE for those people that the LLM output is better than what they could make, plus also that thousands of people sometimes flock to read it.

However there is a problem with the particular "why would I play something a human didn't make" argument, in three short parts:

Part 1

• 14 Minesweeper Variants, Dis PontibusCan pad this list with examples like Kultisti's dice game, Dupery, Clues by Sam, Wordle, Mac & CHI
• Daily puzzles
• Roguelikes

Part 2

Nature and emergent gameplay.

Part 3

Whether something is known to be made by machine can bias you.

Explanation

While I'm sure some people can already quit reading this article because the "3 short parts" are enough information, a future article 'Make Your Stories Unbelievably Easy To Understand' is the reason I am graciously giving you the longer explanation today.

Part 1, Explained

People have long, LONG been playing non-human-generated work, before LLM's.

• Dis Pontibus was the first steam game I encountered that created 'human feeling' puzzles even though it was all generated, a great technical achievement at the time.
• 14 Minesweeper Variants is using a SAT solver to generate not only puzzle variants, but hints. In fact people even 'gold the level select' which is an unbelievable amount of levels.
• Daily puzzles and logic puzzles like Dupery, Mimic Logic, etc. are also randomly constructed. I've even implemented things like this before. The way you design puzzles even as a human is by 'forward design' where you put in clues and see how they propagate. I actually find it the least surprising this process can be automated.
• Roguelikes constantly give you random scenarios and/or card draws to work with. You pick from a random selection constantly.

You can say, well, a human wrote the variant rules, the relics and card effects, etc. But there were not hordes of people calling the inventors of 14 Minesweeper Variants slop for not making their own puzzles.

Really looking at this I find it not possible to reconcile the huge contrast in these views. Where procgen is simply 'not perceived' as being cost-cutting for human designers. No, this actually shows more that LLM's in this context have guilt-by-association because AI is used for video deepfakes and other spam/scams and have more of a tie-in to people losing their income sources, but AI Dungeon/neurosama/pre-scam uses of GPT do not have this feeling.

One totally true thing though is watching an 'art spam machine' can devalue your art and purpose of creativity, even if this art is in the form of puzzle design, I hope to elaborate on that in the part 3 section.

Part 2, Explained

A lot of nature is beautiful and was not human made. There's that Jon Blow talk mentioning how puzzles are discovered more than invented, similar to math.

The process of designing the gameplay for this game was more like discovering things that already exist than it was like creating something new and arbitrary.
And another way to say that is that there was an extent to which this game designed itself.

Roguelikes and CCG's frequently have emergent discoveries and combos beyond what the designers predicted.
Speedruns of games are heavily complex and use unintended glitches but that still have incredible stories nonetheless.

Part 3, Explained

A pro-AI point I have heard occasionally is "if [your favorite creative work] was made by AI, or vice versa, would you think it is less valuable?" - this is actually the fallacious framing of the argument even though it's common. This is because AI-created work has certain choices that go out-of-distribution of a human work and vice versa, so it's really not true that you could just 'say' that a human work is AI or vice versa.

Regardless the best framing is about having a consistent quality-focused view, to not just call something 'slop' because it is machine-made, but only if it is low quality. Much of AAA was already very low ambition work.

It is also not a common ability to be able to tell AI writingNYT AI Writing QuizAn interactive New York Times feature testing whether readers can tell the difference between human writing and AI prose. (they mostly can't)nytimes.com or AI artAI Art Turing TestScott Alexander's experiment challenging readers to distinguish between human-made art and AI-generated images.astralcodexten.com if not explicitly told. So just that small bit of info can heavily bias your perception:

SPOILER
Click to reveal

« Prev 1 / 1 Next »

Even this picture is used a lot but the site is all-anonymous so...

Most recently on my latest videowhat you may not know about these game combination videosIcely Puzzles - YouTube - my video calling out a lot of low-effort AI bad things, but also emphasizing that my perspective on it isn't fully antiyoutu.be I got messaged from a different developer about an upvoted comment about the Super Mario 64 + Minecraft mod, according to them this mod is very likely pretending to be handmade. The comments on the suspected modI Modded Minecraft Into Mario 64Swirly - YouTube - suspected as being also AIyoutube.com are very divided on it, but there are tons and tons of comments complimenting this work for not using AI. This could easily become yet another example in the future.

This "thought of as higher quality because of outside perception" was also mentioned in my Framing article:

There's things like highly valued forgers of fake Vermeer paintingsHan van MeegerenAn article detailing the life and Vermeer art forgeries of Dutch painter Han van Meegeren.essentialvermeer.com that were appraised highly and then lost value afterwards.

Even authorship before this era could be muddy involving, asset packs, public domain work that's been cut up and chopped and reused. When I played games as a child I never very much questioned 'who made it'. Sometimes it is the whole point with ghostwriting and collaborative works with unclear authorship. (Although nowadays I'm more skeptical of this too.)

Basically, the lens of 'human authorship is credible for quality' already wasn't always true, you can find 'interest' in AI work if you don't notice it, or you can be tricked into perceiving a human work as slop because someone told you it was AI. This is reminiscent of when AI (as in computer-opponents in games) beat humans more when they are hidden because then players couldn't try to exploit it. It makes you wonder how hyperstitional, this perception of quality is, especially because game popularity is such a snowball compounding effect.

Efficiency vs. Purity Work, Divide in Society

At this point there are like 12 or more ideological divides in society and people's views, but one at the core of this machine-generated topic is how much creatives should go through the full authentic version of their experience to make the art, and that there is also inherent value in the experience.

This is a very pure ideal, but as you know from existing, the ones at the top often successfully got ahead by automating everything. Do everything yourself and you cannot exist in the current economy.Walkthrough CultureI often see people use hints and walkthroughs for puzzle games, even using the unbelievable turn of phrase "I had to use it". 9-23-2026 Due to wealth inequality some creatives are only thousandaires. And back then proto-language neural networks as translation technology was thought of as a "great equalizer", very different vibes to AI at the moment.

They would also point out this efficiency thing runs deep in everyone's usage of interacting with the world though, certainly you are using computers, a very automated machine, built by factory, you didn't build all the parts yourself, also you're typing letters and words not invented by you.

This also combines with the "games are not just one thing" view, and people without successful projects are like doing 10 jobs without getting paid for it. I've found really small indie devs where they literally cannot do art and UI. (For example the New New Chess Battle Advanced creator.) I feel like I've done more for creatives than others just for improving such projects with better UI.
The 'value in bad art' feels like the way family/friends can give encouraging words to something you're not good at, where this do-everything-yourself becomes sabotage. Then those same creators will get roasted or ignored by outsiders in their earnest attempt (see the fps game 'incursion'). I remember I tried legitimately to do fiction writing 8 years ago and got very insulting "what the hell is this" type responses. People will ignore your vision and core intent.

Humans Being Special Compared to Machines

Were we fundamentally mistaken about chess, etc. being human pursuits?

Yet another divide is humans having a special taste that cannot be replicated by machine.

Chess vs. AI

• Many people in the past premised that it would be harder for machines to figure out systems 'designed' to be difficult, like Chess and Go, instead of things like the physical world.
• Before "AI" got vibes of spam and deception and other ugliness, much due to human misuse, AI in the form of chess engines existed.
• Some believed AI 'solving chess' would make Chess unplayable. It is not yet solved literally, but practically the best engines have been significantly better than any human for a while.
• The chess world still goes on basically ignoring the engines, of course.
• That highlights how much 'stories' matter for this kind of thing. That chess game happened 'between two people' and their playstyle, mindset, context, past, all of this is what people get emotionally invested in and find meaningful. Watching an AI game, there are bizarre seeming empty moves, positional plays that only pay off 10 turns in the future that often violate an almost-always-true and reasonable chess principle you would teach to a beginner, and these can be insane to look at, but they don't have a story (what are you going to look at? if else tables? inscrutable multiplication matrices?), and no chess channel would take over the channel with games of just Stockfish vs. Leela.
• Sort of strangely, LLM's (which can do thousands of things more than just chess) with far weaker, hallucinatory moves, can have more 'story' than these chess engines, though not anything like human stories.
• As it turns out AI has more trouble interacting with the physical world than abstract strategy games.

https://franklantz.substack.com/p/the-afterlife-of-goThe Afterlife of GoFrank Lantz's reflections on the impact of AI breakthroughs like AlphaGo on human mastery of the board game Go.franklantz.substack.com

There was a special aura around Go that made people think it might be especially hard for AI to master. The search space was so large that mechanical, brute-force methods couldn't plumb its depths. You needed subtle intuition, ineffable pattern-awareness. And maybe good old fashioned monkey brains had some magic in these departments that silicon couldn't match. Obviously, this turned out not to be the case. In fact, in the age of transformer models, we've learned that AI is very good indeed in these departments.

It's interesting we saw novelty and uniqueness in rigid things like deciding to sink our minds into 8x8 chessboards of rigid movesets. Watching a chess game had a 'humanity' to aspects like a queen sacrifice or knight jump, even speedrunning and turning yourself into a machine to execute the exact same actions to be more consistent at the task, is looking very ironic right now.

Conclusion

Let's look at it from the perspective of one of those puzzle games I mentioned earlier that use 100% generated puzzles, even if rules are handmade. Then the 'core part of the gameplay' is not within the specifics of the puzzles or scenarios. And this whole thing involves disagreement among many people of what is considered to be 'core of a game'.
For example I watched the game "Play Lethal", a game inspired by CCG puzzles, where people discussed the AI art more than the gameplay, so the art is more core than the gameplay for those people. When this happens I do wonder how much "do you support creatives" really applies for them.

I actually find "AI balance" (a concept pre-LLM's) overall worse than AI art in terms of its effects on the quality of the game, and the 'trust the stats' methodology for example, the fun Old Old Chess Battle Advanced is going to be ruined with the developer's planned AI balance. No one cares though.