Basic Software Development Practices
Engine development at its core is a software application, which requires software engineers to develop. The team has had a solid focus on the game development side of engine development, however we are all junior developers and still learning best practices of basic software development. Although we successfully developed a game engine, we struggled with some of the basic software development practices that engineers in industry have locked down: testing, commenting, formating, and version control.
Testing an engine can manifest in different forms, with tech demos and games, however at the core should also have the typical testing framework used in software development: unit and integration tests. We did unit tests... If we said that to a professional, then showed them our testing, we would be laughed off the stage. We definitely didn't have full-coverage, it would even be bold of us to say of us to say we had partial coverage. For us there simply wasn't time to develop a strong testing framework, however that isn't to say there wasn't the need.
Our main foray into testing was unit testing our math classes, however we didn't follow test-driven development, writing the tests before the actual implementation. We didn't have a strict policy or a process at all of determining what should be tested. This lead us to not writing tests for some of the methods that actually had errors in them, mainly because these were the harder methods and classes to test. The reason we were eventually able to fix the errors in those math classes was from github users poking through our code and opening a issue or because we were fighting a bug that could only be caused by a math problem. These bugs were the worst to face, because we had the fake confidence that our math was tested.
What we learned was that testing was almost like a full-time job for the project. But that isn't completely accurate to say either, as it is more akin to a way of developing, one developer can't be solely responsible for testing. The team needs to have the mantra of testing, the code needs to be developed with the thought of testing, it is a way of developing not just a part of it. For us, we were too enamoured by developing features that we weren't going to be slowed down by testing. We often went weeks without running the few tests that we did have! This even deterred us from creating tests, as our test framework had become out of date and required large amount of work to get working again, just for tests.
We were asked whether the project would have benefit from a stricter testing phase to developing, with each code iteration requiring some testing; and the answer is no. While our engine could have used more testing, the projects mission was to learn and we wouldn't have been able to learn as much by being more methodical. Testing would have required us to remove features which would have produced a stable engine as a product, however we have no care for a stable engine (and admit the Isetta Engine isn't). What the Isetta Engine allowed us to do was see the entire picture of engine development as a whole, what it takes to build one, and the engine development process.
If we were hoping to ship the engine or have users really use it for a project (not just a game jam), we should test the codebase. This also ensure the engine doesn't regress. However, if you want to learn engine development to learn, full test coverage just isn't necessary.
Commenting, every developers favorite part of development... At best, we can say we tried to comment but really it was something that we were just so unenthusiastic about that we all let each other slip by. What we didn't expect when we were trying to comment was how much effort and time it actually took to write them. This is something we thought may lose an hour of development a week to not multiple. However the week we started commenting was the week we lost an entire week to comments. No features or systems were developed during that week. And we weren't even commenting our
cpp files, only the header files public variables and functions! Additionally, what we hadn't factored into this was how much iteration and refactoring goes into an engine. The comments we wrote that week quickly became obsolete, requiring them to be rewritten, that's just time wasted for us!
If we continued down the path of commenting the engine, the engine simply wouldn't be complete. We saw the benefit to comments, for the hypothetical game developer of the engine and the others who might be interested in learning from the engine code, but we had to admit it wasn't feasible for us. We attempted commenting one other time, prior to our game jam, which taught us game developers don't care about the functions that you have commented only the ones that aren't. What we mean by this is that as soon as that they find something uncommented, they no longer trust the engine to provide any information to them. On top of that, we found that our game jammers were much more interested in sample code over any comment.
The way in which we used comments effectively was in the form of
TODOs. When a feature needed to be developed, but not necessarily for our target game we would place a
TODO comment there. We hope that these comments can help encourage curious engineers into trying to develop a feature with the Isetta engine. However, even with these
TODOs, there are far more than we ever intended to have and most lack a good task description to be useful for someone who might not be familiar with the system already. We almost used
TODOs as a replacement for failing unit tests, which is probably not how they should be used.
Something that we had received advice on prior to starting development was to establish a formatting guideline, there wouldn't be enough time during development to fight over silly formatting practices. This advice was so right, and luckily we chose a linter which would automatically format our code. However, if it ended there we wouldn't be speaking about it here. Although we had chosen a linter, there was nothing (no one) to force us into this styling. Frequently all members of the team would push unlinted code or have a weird inconsistent styling with the rest of the engine because it was convenient at the time. This also corresponds heavily with our interview with Casey Muratori about a programmer's native language; we each had our own way of writing code and trying to write it in this rigid, formatted way was difficult for each of us. The only thing we can recommend to others is have a style that your team can agree on and when you find exceptions to that styling be consistent about it; but mainly don't waste time over formatting arguments, especially when time is constrainted.
Our struggle with version control can be traced more specifically to Git. Each version control has its own unique, best practices and it took us awhile to establish a practice using Git that worked for the team. And we still feel it was inadequate. We had all used Git previously and were familiar with it enough that we initially weren't concerned with using it. However, it has possibly caused the most headache and time waste when compared to any engine system.
We were using the process of Git-flow, however had ulterior motives for our branches. The branches didn't only dictate a task but rather an entire system of the engine, so that those interested could follow along with a branch if they were interested in a specific feature. This caused us to have an exorbitant amount of branches, with older branches rapidly going out of date. This was only compounded by each developer on the team using Git slightly differently, with some people rebasing, others only merging, etc. which caused a mess of commit messages. In addition to our personal Git problems, we were possibly creating problems for others because we hadn't developed any type of smoke testing prior to pushing to our develop/staging branches. This inevitably caused regression for the engine and frustration for the developer who was stuck with broken code they didn't create. We also experience some weird and unnecessary merge conflicts which we believe came about because of refactoring. The refactors of the engine went beyond changing a single class name, it went all the way to refactoring the name of the engine! This was a massive headache for us to merge our code into because most of that file directory no longer existed by that name. It took us longer than we care to admit to halt any refactors until all development had been "checked-in", which we then did these refactors late in the night/early in the morning when no one else was developing.
No one was here to keep us in check with our software development practices. Our faculty advisers could only advise, and no one on the team knew enough to course correct us. Our skills with these practices increased through the semester, however this could have been somewhere we could have used an experienced developer the most.