Basic Software Development Practices
A game engine, at its core, is a software application, which requires software engineers to develop. The team has had a solid focus on the game development side of things up until this project, and we're all junior developers who are still learning the best practices of basic software development. Although we successfully developed a game engine, we struggled with some of the basic software development practices that engineers in industry have locked down: testing, commenting, formatting, and version control.
Testing¶
Testing an engine can manifest in different forms, with tech demos and games, but it should also have the typical testing frameworks used in software development: unit tests and integration tests. We attempted unit tests... but if we said that to a professional, then showed them our testing, we would be laughed away. We definitely didn't have full coverage; it would even be bold of us to say of us to say we had partial coverage. For us, there simply wasn't time to develop a strong testing framework, but that isn't to say there wasn't the need.
Our main foray into testing was unit testing our math classes, but we didn't follow test-driven development (writing the tests before the actual implementation). We didn't have a strict policy or a process at all of determining what should be tested. This lead us to not writing tests for some of the methods that actually had errors in them, mainly because these were the harder methods and classes to test. The reason we were eventually able to fix the errors in those math classes was from GitHub users poking through our code and opening a issue or because we were fighting a bug that could only be caused by a math problem. These bugs were the worst to face, because we had the fake confidence that our math was tested.
What we learned was that testing was almost like a full-time job for the project. But that isn't completely accurate to say either, as it is more akin to a way of developing. One developer can't be solely responsible for testing. The team needs to have the mantra of testing, the code needs to be developed with the thought of testing; it is a way of developing, not just a part of it. For us, we were too enamored by developing features that we were averse to being "slowed down" by testing. We often went weeks without running the few tests that we did have! This even deterred us from creating tests, as our test framework had become out of date and required large amount of work to get working again.
We were asked whether the project would have benefit from a stricter testing phase to developing, with each code iteration requiring some testing, and we believe the answer is still no. While our engine could have used more testing, the project's mission was to learn, and we wouldn't have been able to learn as much by being more methodical. Testing would have required us to remove features which would have produced a stable engine as a product, but engine stability was actually not a goal of ours (and we admit, the Isetta Engine isn't!). What the Isetta Engine allowed us to do was see the whole picture of a game engine, what it takes to build one, and how the engine development process works.
If we were hoping to ship the engine or have users really use it for a project (not just a game jam), we should test the codebase. This also ensure the engine doesn't regress. However, if you want to learn engine development to learn, full test coverage is a harder thing to bargain.
Commenting¶
Commenting: every developers favorite part of development... At best, we can say we tried to comment but really it was something that we were just so unenthusiastic about that we all let each other slip by. What we didn't expect when we were trying to comment was how much effort and time it actually took to write them, especially if we were also using a documentation engine like Doxygen. We thought we'd maybe lose an hour of development time a week for comments, but the week we started commenting was a week that was entirely lost to comments (see if you can find it in our weekly blogs!). No new features or systems were developed during that week. And we weren't even commenting our implementation files—only the public header files! Additionally, what we hadn't factored into this was how much iteration and refactoring goes into an engine. The comments we wrote that week quickly became obsolete, requiring them to be rewritten, and that was just time wasted for us!
If we continued down the path of commenting the engine, the engine simply wouldn't be complete. We saw the benefit to comments, for the hypothetical game developer of the engine and the others who might be interested in learning from the engine code, but we had to admit it wasn't feasible for us. We attempted commenting one other time, prior to our game jam, which taught us game developers really don't care about the functions that you have commented, they only about the ones that aren't. That is, as soon as they find something uncommented, they don't trust the engine to provide any information to them. On top of that, we found that our game jammers were much more interested in sample code than any comment we wrote.
The way in which we used comments effectively was in the form of TODOs. When a feature needed to be developed, but not necessarily for our target game, we would place a TODO comment there. We hope that these comments can help encourage curious engineers into trying to develop a feature with the Isetta Engine. However, even with these TODOs, there are far more than we ever intended to have and most lack a good task description to be useful for someone who might not be familiar with the system already. We almost used TODOs as a replacement for failing unit tests, which is probably not how they should be used...
Formatting and Style¶
Something that we had received advice on prior to starting development was to establish a formatting guideline because there wouldn't be enough time during development to fight over silly formatting practices. This advice was so right, and luckily we chose a linter which would automatically format our code. However, if it ended there, we wouldn't be speaking about it here. Although we had chosen a linter, there was nothing (i.e. nobody) to force us into this styling. Frequently all members of the team would push unlinted code or have a weird inconsistent styling with the rest of the engine because it was convenient at the time. This also corresponds heavily with our interview with Casey Muratori about a programmer's native language; we each had our own way of writing code and trying to write it in this rigid, formatted way was difficult for each of us. The only thing we can recommend to others is have a style that your team can agree on and when you find exceptions to that styling, be consistent about it. Don't waste time over formatting arguments, especially when time is constrained.
Version Control¶
Our struggle with version control can be traced more specifically to Git. Each version control software has its own unique best practices, and it took us a while to establish a practice using Git that worked for the team. However, we still feel it was inadequate. We had all used Git previously and were familiar enough with it that we initially weren't concerned with how we would use it. However, it has possibly caused more headaches and time waste than any of our engine systems.
We were using the process of Git-flow, but we had ulterior motives for our branches. The branches dictated not just a task but rather an entire system of the engine, so that those interested in a specific feature could follow along. This caused us to have an exorbitant amount of branches, with older branches rapidly going out of date. This was only compounded by each developer on the team using Git slightly differently, with some people rebasing, others only merging, etc., which caused a mess of commit messages. In addition to our personal Git problems, we were possibly creating problems for others because we hadn't developed any type of smoke testing prior to pushing to our develop and staging branches. This inevitably caused regression for the engine and frustration for the developer who was stuck with broken code they didn't create. We also experienced some weird and unnecessary merge conflicts which we believe came about because of refactoring. Several of our refactors of the engine went beyond changing a single class name—sometimes it went all the way to refactoring the name of the engine! This was a massive headache for us to merge our code into because most of that file directory no longer existed by that name. It took us longer than we care to admit to halt any refactors until all development had been "checked-in", which we then did these refactors late in the night (early in the morning) when no one else was developing.
No one was here to keep us in check with our software development practices. Our faculty advisers could only advise, and no one on the team knew enough to course correct us. Our skills with these practices increased through the semester, but this was somewhere we could have used an experienced developer the most.