September has begun, and we start getting natural questions like: “When will the open-source server go live that was promised in August?” So in this post we’d like to explain the situation that’s going on now.
Currently we have actually three parallel development pipelines for Screeps:
- Features of the game itself and its gameplay
- Optimizing the official server
- Development of the standalone hosted server
When our second Indiegogo campaign was launched, the plan was roughly this: make a market system as a major feature, and then delve completely into the development of the standalone server and a Steam client for it. That’s why we set August as an approximate milestone date. However, after the campaign’s end and some consideration we realized that the Steam launch proper and the standalone server release are two vastly different things that should be better done one at a time rather than simultaneously. A higher traffic from Steam may require some adjustments in the scalability of the game servers, so we’d be having hard time watching this process and at the same time taking care of the open-source project that would appear along with it. We decided to first launch the Steam client to attract users to the main official world, debug it, and then move to the standalone server without any hassles.
Our experience has shown it was a right decision. When Steam gave us hundreds and hundreds of new players, it revealed a number of issues in our server architecture that prevented capacity scaling for running your scripts, and we put a lot of time and efforts to debug and optimize those bottlenecks. You might remember how the tick duration soared from 2.5 to 5 seconds, then fell back, then grew again, and then stabilized at 2.7-3 seconds. We are content with the current rate, but achieving it required a lot of work within the past two months.
Due to this reason, unfortunately, the standalone server won’t be launched within the coming weeks. We are sorry to realize that we missed some deadline, but please be assured this was not due to our laziness, but we decided to rearrange the work, and the history proved it was a right decision.
And there is another reason for delaying too. After the Indiegogo campaign was over, we spent some time on designing the concept of a standalone server, and we realized we couldn’t just take the current server code, open-source it, and rest on our laurels. For the project to gain traction as standalone as we see it, the following design concepts need to be implemented:
- Zero dependencies. The official world works on a traditional server stack of many applications: nginx + nodejs + mongodb + redis. And it won’t work for the standalone server. The user should be able to just download an executable file via Steam, launch it, and get a working server at his IP address without any extra programs to install. We use Electron packaging to achieve this, but it also requires a separate database layer based on the in-memory database inside the application itself.
- NPM-based application. Though many users will opt for the easy downloading and running the server as a simple desktop application, we have to support a more advanced installation as an NPM module as well. Being a JavaScript application, the Screeps server organically fits into the npm packets ecosystem, but the current game server architecture (which was initially developed as a set of layers with loose coupling) should be divided into multiple modules to publish them in the npm separately. This will allow, for example, replacing the database layer module with your own one to use the desired server-side stack.
- Fully moddable architecture. To change the majority of server behavior aspects should require simple redefining of needed fragments through a system of external hooks and extensions rather than modifying the server code. We also need a possibility to add custom game entities even with their own visuals.
- Single-player bot players. The single-player mode (which makes some people long for the standalone server) will be pretty boring unless there is somebody else but the player in the game world. But the development of a full-fledged AI that can start from scratch in the game world and seriously challenge the human player is a very interesting and complicated task. We want the players themselves to find the solution to it, since that’s exactly the main fun of Screeps. Any player could write a script for his or her standalone AI and submit it into the common repository where other players of the single-player mode can install it. When provided with its own room in the worls, such a standalone AI has to be able to analyze the situation, develop its colony, and bring some life to the game world where it exists. The repository will introduce a ranking system for AI performance, user comments to them, new version updates, etc.
- Steam Workshop support. The two aforementioned options are perfect in terms of excellent mechanics of Steam Workshop. The system of downloading and installing any custom files provided by the Workshop can be utilized to automatically install and update custom mods, game object types, bot players, and entire complete worlds. This is a huge task, but it’s vital for the Screeps ecosystem.
Implementing some of these things requires the overhauling of the current server code of Screeps since they were not accounted for 2 years ago when the game was first designed. But we are convinced that short of these, the standalone server won’t be the type of product we really want to share with you.
In terms of deadlines, we don't want to be overly optimistic at this point making promises we won’t be able to meet. Let’s keep October in mind. If we get a working version earlier, we’ll definitely show it. As was promised, our Indiegogo campaign backers will be the first to get their hands on it, while the public availability will begin a month later.