Grumpy Gamer

I was having a discussion with someone on Mastodon about unit testing games and how it is a next to impossible job. Once clarified, I think we saw eye-to-eye on it, but I do hear about it a lot, mostly from programmers that don’t live in game dev.

Testing games is hard. So much of what fails during testing is due to random behavior by the player. Them doing something you didn’t anticipate.

I break testing down to three groups.

1 - Unit testing

This is what I hear about the most and how this would be a good way to test games. It is not. Unit testing will test code components of your game, but not the game. If you have a sorting routine, this will test that, but it has little effect on testing your game.

I do unit testing, mostly on the engine commands. This get run once a week or when I add a new feature. It’s testing that all the command or functions return correct values, but has little to do with “testing the game”.

It’s also something that would be very hard to run from the build process since the entire engine needs to start up. This is why I run it by hand every so often. If it’s not being run in a real engine environment, it’s not accurate.

2 - Automation

I’ve been using automation since Thimbleweed Park.

I called it TesterTron 3000 it ran through the game and randomly simulated clicks and tried to follow some logic. It found a few things, mostly bugs that would only happen if you clicked very fast.

It was fun to watch and gave mostly peace of mind testing. It often broke because we changed the games logic and it could no longer follow along. If I had a team that did nothing but keep TesterTron 3000 working, it would be more useful but given the limited resources of an indie team, I’m not sure it would have been worth it.

I have something similar in Death By Scrolling, it runs through the levels and randomly picks up things and try to attack enemies.

I could have it follow a set sequence of key strokes, but then it’s only testing what you know works, not the goofy stuff that real bugs are make of. I’ve run into programmers that build something like this and at first it feels good but as the game changes it falls apart and doesn’t get used much after that.

Maybe if you had a simple puzzle game, this might be useful.

TesterTron 3000 a good stress tester and it needs to run overnight to be truly useful.

3 - Human Tester

The most important testing we do is with testers who play the game all day long. They look at the Git history and see what has changed and beat on that. 99% of our bugs are found this way and I can’t emphasis enough how important it is.

Players do odd things that automation never will.

While it is important to test your own code, you will never find the “good” bugs. Programmers are lousy testers because they test what they know, you need to be testing what you don’t know. Be afraid if management that says they don’t need testers because the programmer should test their own code.

I grew up in a world where a bug meant you have to remake a million floppy disks and it would take months to get the change out to players, if they ever got it.

We just got Death By Scrolling running on the Switch. We had to do some optimization because we were doing stupid stuff with rendering the tile map. Modern PC machines are so fast these days that you can do a lot of stupid stuff and it doesn’t matter. Back in the day (queue angry old man) I had to count cpu cycles and every byte of memory was precious. Nowadays memory is basically infinite.

But it’s different for consoles, they do have a limited amount of memory and you do have to pay attention to performance.

One thing that really bothers me when I play Switch games ported from PC or other consoles is the lack of care over font size. Things that look good on a big monitor or TV are unreadable on the Switch (and the Steamdeck).

We’ve taken great care to make everything big enough to be readable on handhelds, but it’s a real pain. As a designer you want to cram enough information on each screen, especially for RPG-ish games.

People get used to html/css rendering and how good it is at flowing to fill space and around images. Games often don’t have text rendering engines that are that complex (using a html/css engine internally is overkill).

One of the big upgrades I want to do to my engine is better text flow rendering, it will never be as good as html/css in the browser, but it could be a lot better/easier. It’s one of the things I’m envious of Godot.

If you haven’t read my previous post about Death By Scrolling way back in February, I suggest you do.

Of course this is my lazy way of doing the 2nd promised blog post for Death By Scrolling.

In all fairness, I started to write it and it seem awfully familiar so I went back and checked and sure enough I had already written about it.

But, I’ll do another real post…

I asked for beta testers on Mastodon and got close to 300 sign-ups. I didn’t want to invite everyone all at once. There is an old saying that you can only make a first impression once.

Every time I make a new beta version I invite 25 more people.

Couple of stats:

About 25% of the people never redeem the steam key. Or they redeem it weeks later. This is a little surprising, but maybe it shouldn’t be. People are busy.

Of the people who did redeem the key a third play the game once or twice and never again. This is not surprising. Death By Scrolling is a rogue-like and you die a lot. I do mean a lot, it’s right in the title. Some people do not like this type of game, and I’m OK with that.

Maybe half the people who play the game never visit the Discord. We can get only so much info from analytics. Having a conversation about what you like and don’t like is very helpful. Again, this isn’t too unexpected.

The players that do play more than a few times play a lot and that is good to see. It’s nice to see strategies emerge that we, as the designers, didn’t think of. That is always a good sign.

This is the first time I’ve done large-ish beta test for one of my games and it’s been fascinating and very insightful.

I’m about to invite the next group of 25 testers. If you’re among this group, please visit the Discord.

– Ron

I miss the Kickstarter days of Thimbleweed Park where each week I would write a blog post about how things were going and we’d to a podcast. If we were doing Thimbleweed Park today we’d have a video podcast on YouTube.

I watch a lot of YouTube videos from indie game devs where they document everything they are doing on their game with a fancy video and my first thought is: “Where do you find the time to make this”. I barely have enough time to work on my game.

Now that Death By Scrolling is getting close to releasing, I really should start blogging again about the game. There are a lot of interesting things happening and there is no reason I should keep those to myself.

I have found that as I get older I enjoy pontificating less and less. I’ve stopped doing talks, doing interviews and podcasts. I really enjoy Tim Cain YouTube channel but I could never do that. It’s not that I don’t have anything to say, it’s that I have become self-conscious that I’m just pontificating for no reason and don’t have anything deep and interesting to say.

Now that I’ve got you all primed and ready, starting next week you can plan your day around the Death By Scrolling blog update.

Or maybe I’ll delete this post like it never happened.

^^^ this

When I build my game for testing, it’s a completely automated process from the command line.

I type pub.sh --test on the command line and a long chain of events is started.

The script pushes to git, which starts the cloud based CI machine (Azure) building the Mac, Windows and Linux executables.

When they are done, my local script is notified and they are download locally.

Game files are added and they are signed and packaged up.

The script then logs into Steam and uploads the game.

A change list is built from the git log and uploaded to a private website.

When that is complete, a message is sent to Discord to notify testers that the build is done.

This is all done with no interaction on my part.

It makes doing test builds quick, easy, and painless. But more important is there is no human interaction, so there is less chance of a mistake being made.

To upload to Steam you need to authenticate, which usually involves getting an email and typing in a code or doing the same with their authentication app. It’s not very conducive to automated builds.

Fortunately Steam has an process where you authenticate once and then pass a token from the command line and that logs you in without a password. Wonderful.

This worked fine for what seem like years, then on Jan 1, 2025 I got a error saying my token was expired. OK, makes some sense.

I re-ran the process to get a new token and everything was fine for a day or so, then it failed again saying the token was expired. Then it happened again. And again.

Frustrated, I reached out to the friend who works at Valve and was put in contact with a programmer there. He watched the back-end and said everything was fine for the first few auths and then the old expired token as submitted. Rebuilding the token worked for a while and then the old token was submitted again.

We began to suspect the token file on my machine was being overwritten but I couldn’t see it happening and didn’t know how it was happening. It not like the whole directory was being restored from a backup, it was just that one file sitting in an obscure Steam folder.

We tried everything and never solved the problem.

The solution I finally arrive at – and has worked fine for the past three months – was to get an auth token and then save that file away. Just before my script uploads to Steam it copies that good file over the current one and uploads. This is what Steam recommends for full CI builds, but until Jan 1, I never had to do this locally.

I’m still mystified who is overwriting that file but I guess it’s moot now.

Going on n+1 years, I proclaim grumpygamer.com to be 100% April Fools’ Day joke free.

I realized that I lost a large chunk of the early April Fools’ posts due to moving content from one back-end to another over time. It pains me as I’m sure they are seen as culturally significant and worth preserving for future generations.

“Grandpa! Tell us the story of when the web used be be April Fools’ joke free!”

I might have to go track them down on the Wayback Machine and pretend it all never happened and I have perfect back-up and archiving practices.

First screen shot from my new game called Death by Scrolling.

I started working on this game back in 2019. It started out great and everybody I showed it to loved it, but it was simple. Everyone asked for progression and “game” stuff and over the next six months I proceeded to ruin it.

Then a little game called Return to Monkeys island came along and I took the next two years to build that with Dave Grossman and a wonderful team.

Monkey Island ended and I started working on a new game I dubbed “RPGTBD”. It was a classic Zelda-like pixel art RPG and was a lot of fun. I brought on an artist (Craig) and a designer (Elissa) to help.

When I first started RPGTBD everyone told me that it’s unrealistic to do a open world RPG with a small team. Morons. What do they know?

Well, a lot actually.

It became obvious that my vision for the game was never going to get done and on top of that I was spending a lot of money so something had to give. I could pay off the contracts for Craig and Elissa and call it a day.

But I had all this wonderful art and a nice quest and exploration system from RPGTBD it felt odd to let them go to waste.

Then I remembered Death By Scrolling. I shared the prototype with Elissa and she really liked it. There is something about the game that I could never let go of.

I rebuilt the whole project, stripping out all the stupid stuff and returned it to the core of the game back when it was fun.

Over the next 3 months Elissa and I added back pieces bit-by-bit with a nice progression system, story, quests and challenges.

It’s not going to be everybody’s cup of tea, but Elissa and I get lost in playing it when we should be testing a new feature, so I guess that’s good.

ELissa and Craig leave the project on March 1 then it’s just me for a few months and then release it in 2025.

This time for sure.

I recently asked on Mastodon how a player would expect damage to be computed in an RPG.

(base_damage + extra_damage) * damage_buff

or

(base_damage * damage_buff) + extra_damage

General consensus is there was no consensus.

The one thing people did agree on was that the skill/weapon description should say what the order of operation is.

Fair enough.

I am worried about the amount of information that needs to be conveyed to players. The game needs to work well on the Switch and the Steam Deck with limited screen size. I play a lot of games where the 20-something devs with perfect eyesight spent too much time looking at the game on a large monitor. I’ve purposely kept the text large enough to read on a small screen. It means I need to be succinct.

Some people suggested that players should be able to order the weapons/skills as they wanted. This is problematic since weapons/skills come in a very random order and a UI to reorder them seems like a lot of work and not really what the game is about.

It also doesn’t matter what order the weapons/skills are evaluated. I compute damage_buff then compute damage_extra when the weapon/skill is acquired. Later on those are used to compute damage and that is where the order matters.

It’s also basically an action game and players shouldn’t get too deep into stat management for a character that is going to last a few minutes.

Maybe I’m wrong about that.

I think I will opt for the 1st method since it allows for bigger numbers and that is something players do want.

I’m a big fan of Tim Cane’s YouTube channel.

If I was going to have a YouTube channel, I would want it to be like his but talking about adventure games, not RPGs. Maybe someday.

So imagine my delight when he posted this:

Clayton and I wrote this poem many years ago, but not much as changed.

I’ve never met Tim Cane but hope to someday. Maybe he will come to New Zealand.