Engine Roadmap
Apr 07, 2015
The following is the roadmap for the Thimbleweed Park 2D point & click engine. Keep in mind this is just for the engine, not for the actual game.
I'm really happy and excited by the progress on the engine so far. I'm much farther along than I thought/feared I'd be at this point. The original plan was for David Fox to come on in June to begin game scripting, but I think we're going to move that up by a month. Around a quarter of the game is already walkable, so it's time to start wiring up some puzzles.
All of the engine tasks lived their lives solely in my head, but I figure it's time to start writing them down and making some task lists and put together a roadmap of where it all is going.
In the early stage of a project, schedules are more guidelines than ridged dates and milestones not to be missed. I like to get everything down and listed, but be flexible enough to move tasks around as needed. I tend to focus on risky endeavors first. If I don't know how to do something, I like to get it done early, at least to the point where I understand how to do it.
I also like to hit everything with broad strokes, get all the system working at a basic core level, then go back and add features, work out kinks, optimize, and generally make them look better. It can be a real trap to spend too much time making one system look perfect, only later to find you don't have enough time for a more important feature, or worse, you end up cutting the system you spent a month polishing too early.
The actor scaling in the last post is a good example of this. I'm not happy with the way it looks, but I wanted to get scaling in and working, it's important for the room layout's Gary is working on, and there is also a lot of code infrastructure related to scaling, like how to specify the scaling, linking actors to scaling, etc. Now it is working, I'm going to move on to the next item. I'll revisit the visual aspect of the scaling a few months from now. How it looks isn't important right now.
Thimbleweed Park is going to be an ugly game for many months. That's how it is. We're posting raw un-censored, un-pr-sanitized images and videos.
April
Sound system and basic commands to play sounds - I'm currently just using SDL_MIXER for audio and based on using it in other projects, it seems to do everything I need. I've looked at solutions like FMOD, but they are too expensive, even with their "indie" pricing. We'd end up spending over $6000 just to license FMOD for Win/Mac/Linux, I'd rather use that money on actual music. I might drop in a better system down the road, but this gets us started.
Scrolling inventory - The inventory is working, but you can't scroll it yet. It's not an issue until you get more than 8 items.
Windows version - I want to get the engine compiling on Windows. I had my GGLib compiling a test version of Scurvy Scallywags on Mac and Windows, so I image getting Thimbleweed Park compiling and running on Windows shouldn't take more than a day.
Walk boxes - The editor part of the task is done, I just need to add it to the game engine. I started doing this a couple weeks ago, then abandon it due to my solution being too simplistic and not able to handle turning boxes on and off during gameplay. Plus, I really suck at math.
Actor talking animation - Actors can say lines of dialog (as you've seen), but there is no animation. All the animation is currently broken up into heads and mouths, so it's just a matter a triggering them. Not sure of we'll do lip-syncing. Our mouths are 2 or 3 pixels, but it might be nice when playing audio. That's a "down the road" task.
ThimbleScript Wiki - I'm putting together a wiki of our scripting language, I should have that ready by the end of the month.
Script control over text and image sprites - I need to add commands so the scripts can create and display standalone images and text. There is already commands for showing and manipulating objects that appear in rooms, but scripters will need to create other images and text, mostly for closeups like an elevator panel or a newspaper closeup. All the engine code exists, it just a matter of hooking it up to scripting commands.
Basic cut-scenes - Cut scenes are a little more complex than just turning off the cursors, they need to track where the game was before a possible "cut" to a new room and silently return everything back to normal. Not rocket science, but it will take a good day or two of testing all the edge cases.
May
SFX triggers in animations frames - I'd like to embed sound cues into animations so game scripters don't have to hand time sounds when animations happen. The obvious example of this is foot steps.
GGPackfiles for test builds - All the game assets will be packed into an uberfile. Shouldn't take more than an hour to enable this. It was working before, it's mostly writing the bash scripts to gather up the assets.
Add a toolbar to the debug window plus hotkeys - The debugger window window could use a toolbar and some user configurable buttons. To pause and step execution, you need to use the keyboard, which is fine, but they could use some fancy buttons as well.
Macro preprocessor for Squirrel - Squirrel doesn't have a preprocessor stage, so I'm going to add the ability to do #define, #if, #etc commands.
June
Music system - As I mentioned above, I'm using SDL_MIXER for audio and it is a good fallback, but I'd like something better for music. I'd love to use FMOD, but it's just too expensive and I haven't found anything like it that's significantly cheaper and also cross platform. We need something that will work on Mac/Linux/Windows as well as iOS/Android and consoles. Maybe I'm missing something obvious.
Shaders to test out material system for layers and objects - I want to use shaders to some some fancy-pantsy stuff like rippling water, depth of field, and lights, but still keeping it pixel faithful. Shaders are fun to play with.
Fix window scaling - The game window is currently hard created at 1280x720 and I want get window sizing and full screen mode working again. This was working during the Scurvy Scallywags tests, but I broke it.
Save/load system - This one scares me. If you want to do a "perfect" save game, then it's a lot of data to save from a wide range of places and systems. The easiest thing to do is some kind of "check-point" system, but I wouldn't be happy with that. I want to get able to quit playing at anytime and have the game perfectly restored when I load. I won't go into all the issues that make this hard (maybe another post), but this could take a couple of weeks to get right.
Implementing a complex animation system - I want to create (or find) an animation tool that will allow for the compositing (not necessarily pixel editing) of complex animation scenes consisting of multiple images and timelines. Flash has some good tools for this, but I don't know that it can emit the data in a way that I need and be friendly to 8-bit pixel art.
July
Translation system - implement the system that will pull text from data files for all the different languages. I'm going to implement it the same way we did at Humongous Entertainment and how I did for DeathSpank. It's not hard, it will just take a week or so. The side goal is to allow "user" translations.
Talkie system - Getting actors to speak their lines, plus pull them from different language packs. Not that hard, just a lot of edge cases to handle.
August and beyond...
TDB
- Ron
Reading your posts, it seems you have to build a lot from the ground up (paralaxx scrolling, verbs,...)
The kickstarter video showed an engine that was already pretty complete (compared to MM scumm).
So how come you have to redo all of that? Or was it all just pre-animated and not a real playable engine at the time?
Have to say the best thing till now is you admitting you suck at math ;)) Gives us regular guys the convidence to think we could pull sometihing like this off too!
Enough flattery.... Whats the secret of monkey island??
Anyway,
For example, you'd have a scene in the game that would loop patterns 1 through 9, and could set the number of times to loop. Then if the player travels to a new scene or we randomly fire a cutscene, we'd find our location in the old pattern, if we're at the beginning, quickly fade it out and start the new loop, else we're in pattern 9, queue up A and B to play a segue because that doesn't sound as abrupt. When the scene changes, start patterns C through 1F and put those in a loop. Maybe always have some instruments playing in a muted channel. Inside of the pattern, make a decision whether or not to unmute the channel. Do this randomly to give the soundtrack some variation.
I imagine this is similar in idea to what FMOD and iMUSE do/did, but I admit to knowing very little about either of those.
http://labs.echonest.com/Uploader/index.html
iMuse was amazing to use, but it worked at a very core level, dealing with individual notes. It was also very time consuming. In addition to the musicians, Monkey Island 2 had a full time programmer that did nothing but install iMuse hooks into the SCUMM scripts.
nice updates lately!
I was asking myself: what about dialogs?
Do you plan putting lines and replies in the room code or you plan having a dedicated system for them?
But to answer your question, I would not embed dialogs in code. We did that for Indy, Monkey 1 and Monkey 2, but there are better ways. For DeathSpank I created a special scripting language called .sassy files that allowed for the creation of dialog puzzles without code. If dialogs appear in Thimbleweed park, it will use .sassy files and I'll do a post describing them.
Btw, thanks for the great new blogpost!
Part of this blog is us "thinking out loud", so please understand that.
If you can make the game use a fixed number of objects, you could just give each object a unique identifier (i.e. the name of the Lua global that holds its table) and list of properties that need saving, then loop over a list of all objects and save all those properties out. Since Lua uses tables for each object, you could probably just give un-saveable properties a prefix and then loop over all keys in its table and have your code only save ones that don't have this prefix.
Or am I imagining this as being too simple? I've written my share of file formats and editors, but admittedly I haven't written savegame code yet.
Thank you for the very interesting update, Ron! :D I'm most excited to hear about the translation system. I hope things worked out regarding the font(?).
I also see that you are indeed seriously considering porting TP to consoles since you want an audio program to support such platforms. The way I see it, if you do port Thimbleweed Park to the PS4 and Xbox One for example, those consoles already have close to ten million users each and will continue to sell like crazy for the next 7-8 years to people who usually never play games on computer and mobile platforms. No matter how good or bad Thimbleweed Park would sell on consoles, I just see a giant goldmine of net profit to be made despite the costs of porting and the percentage of sales revenue that Sony and Microsoft would take for every copy sold. And i dunno if you and Mr. Winnick are self-publishing the game, I'm assuming that you guys are the owners of Terrible Toybox, Inc. so other than Microsoft and Sony there really won't be a publisher middleman who's going to take a chunk of the sales away from you guys since you guys are the publishers.
And looking way way into the future, if you and Mr. Winnick do decide to launch another kickstarter campaign for another old-school adventure game running on the Grumpy Gamer Engine or something else, I'm pretty sure we've got your back =) But that's not important right now, there is tons of work to be done on making Thimbleweed Park as good as possible. Mush! Mush! All joking aside, seriously you guys are kicking ass and it's only been 3 months!
I think OpenAL is probably your best bet. It might be a little low level, but it is cross platform and not too difficult to use.
I came across this post the other day:
http://java.dzone.com/articles/crowd-localizing-under-version
As I guess you'll be asking for recommendations concerning crowd localization tools in or about June, I'll just keep that link here as a reference and re-post it then :-D
Maybe I will even have found the time to test it myself by then.
It's great that you intend to make user translations possible above the ootb translations.
I take it you don't store all game-state variables in a common GameState(TM) object?
'Cause, y'know, it'd be just about Marshalling / Unmarshalling then.
Marshalls go along pretty well with Detectives, btw.
...
...
...
Yes, I know. Sorry.
My solution was something like this: I create a SaveData class and placed here every variable I wanted to save, when I called the save() function it serialized the whole SaveData class and wrote it in a file. The load is the same backward with de-serialization.
And if Ron has the same programming OCD tendencies that the rest of us have, he probably wants to use the most robust efficient way possible which will require TONS of consideration.
There is also BASS -> http://www.un4seen.com/
It was really fun to follow the Scurvy Scallywags ones!
The hard part is saving the data in the first place. It's not as easy as it sounds. It's a lot more complicated then just saving a key-value paris for game state. The scripting language is can run a lot of different threads, and each of those threads has their own stack frame and when the game gets saved, a call could be nest quite deep and everything on the stack needs to be saved. Plus the animation system can have many animations playing, and each of those can have several layers, all this their own state that needs to be saved. Then there is the audio. 32 channels. If the SFX is short, probably don't need to save it, but if it's a long SFX, it's position needs to be saved and then restarted in the right spot. Same goes for music. If there is an interactive music system, all that needs to be saved. Then there is patching. A save game needs to still be valid after a patch that could introduce new variables, game states or possible the deletion of data and assets.
None of these are hard on their own, they just take time, but put together, saving is quite a complicated task. I know how to do it, I just dread the task of actually doing it and then debugging and testing it.
Death by a thousand paper cuts.
Q: Do you like a save/Load dialog window?
1. Yes
2. Of course
3. Naturally
Let's vote!
I had actually thought it could be a key component to the game. In addition to your normal dialogue puzzle, I would think that it could also be a way to vary the clues given to the player. Ask the right questions, you get better clues. And, with most characters you should not be able to access the entire dialogue tree during one playthrough. For example, once you reach a certain question (or number of questions), the character would clam up, "You know. I don't really feel comfortable answering any more questions without a lawyer present.", or, "You sure do ask a lot of questions, mister. And, I don't have any more answers for you, anyway.", etc... You may or may not ever become privy to a specific clue (or given a specific item). However, it would take some work to make sure that if you never gain access to that clue that you will be presented with an alternate solution to a puzzle, whether that be easier or more difficult. I could see where a heavily dialogue-driven game could get quite complex, but I thought it would be a great fit for a game about two detectives.
Anyways.
Side note/question about the phone book... Is it safe to assume the Thimbleweed phone book will be a central mechanism for puzzle discovery? For example, you discover a new name or business, you look it up in the phone book to get the address or call them? Will new locations become accessible on the map via the phone book? I assume certain clues will also trigger new dialogues with certain characters... Any special consideration as far as how the phone book will fit into the engine?
It is not fun to do, but I would choose the same dialog over and over again if I played it subconsciously.
Not having dialog trees like in MM, Zak and other 1987-like games would be reassuring!
Or you could think of a clever way to prevent this in the save game system. Or just by design,that unpruned dialog stays available without the need to reload, as in tales of monkey island. Anyway, saving and reloading that often is distracting. I remember just starting all over again each time when playing Zak on the C64, because save/load was long and error prone. Of course most games back then did not have save capabilities, so we were used to that. :)
http://gafmedia.com/
https://github.com/CatalystApps/Cocos2dxGAFPlayer/tree/release-cocos2dx-3
- Spine
- Spriter
(we use the latter). Besides the typical spritesheets you get an (almost) human-readable JSON output that you can use to recreate the animation of the individual parts in your engine. Spine brings a lib for that, with Spriter it's also easy enough.
But if there was a significantly higher number of characters in TP than in MM or Zak, then I would be certainly surprised not to be able to talk to anyone throughout the game. So some few dialogs would be nice. You cold restrict the appearance of those dialogs to the detectives in order to reduce the effort. Just let it depend on the tightness of your time schedule.
You record couple of JSON states in different parts of the game before releasing, and have save games diff against those. They should get pretty small.
I want to ask something midly-related to the translations, and for that i wanna to post in a post you talk about, and maybe it's the best one i think. I don't know if someone ask something related.
You want to add the possibility for user-made translations, this don't affect me directly because i'm spanish but i think it's very good, but what about differents dubs? My girlfriend works in that area and knows some amateurs projects of games that someone want to be dubbed, and they make it real, and although is more difficult than a simple translation, some projects goes to the goal with a decent results. I know it could be (very) difficult in your side, but if you don't decide yet the sound system, maybe you can make something to have the door open to this option of "fan-dubs", maybe having a structure of folders and "exchanging" the sounds of the dialogs or something else.
Only this one. Great work, Ron, always beautiful ;)
since several days i asked myself what was/is your first decision when you start to write a new engine and therefore: how you start, what´s the first class/task you write, on which i suppose much other things will build on?