Dirpack – revived and now available

I mentioned in one of my previous posts that I’ll make the code for dirpack (including my changes) available.

Well, here it is:


You’ll get there both the source code (and CodeBlocks project) and a binary package.




Innocent changes …

You gotta’ love’m.

It’s the kind of mistake you usually make when you’re overconfident. You know, that little edit in the code that is absolutely safe, right? And then the automated build fails because you didn’t bother to compile the code after that tiny change.

Or when you make that change, the code compiles fine. Then when running the app you face weird crashes, bad file formats and whatnot. All because you didn’t bother to test that said change.

So, here’s the story of such a change.

A few days ago I went on reviving an old piece of software called dirpack. I managed to figure out the issue of O_BINARY flag, and I happily compiled the tool. I did another change though that I didn’t bother to test properly. No, I’m not going to tell you what was the mistake, not yet at least.

I moved on with the development for MapTap, knowing that dirpack is compiled and it works (well, it worked when I tested it, but before making that change). So I clicked on the map I wanted to play, saw the .PAK file getting created, sweet! Now to open it with PakExplorer … and dang, the file is not right! The three BSP files were inside a folder called “maps”, and now they are listed at the top level, with a size of -1 KB ….

And then it hit me.

The innocent change I mentioned was changing throughout the code all uses of ‘/’ as path separator to ‘\\’. And you’d expect it to be ok, right? I mean that’s the path separator for Windows, so no harm done, right?


Because we’re not talking about the Windows operating system anymore and how it handles path separators. We’re talking about .PAK files, which is something slightly different. You have to play by the rules imposed by the people who defined how .PAK files work.

And you should always test your code changes. There’s no such thing as an innocent change.

On MapTap – digging up an old piece of software

One of the features that is scheduled for the next version of MapTap (2.3) is the ability to play Opposing Force maps. This is a feature that we thought it was already working, until tests for 2.0 conducted with members from the Steam community pointed out that it wasn’t.

Phillip did a bit of research, called a guy (who knows a guy – just kidding; the person contacted by Phillip was the expert) who knows a bit more about this game, and it seems that Opposing Force is launched basically as a mod for Half-Life. This spells bad news for us, since MapTap would try to launch the proper game with the “-game” parameter, to specify the folder where we unpack the archives installed.

That expert however provided a solution: create .pak files with our contents!

That posed however some challenges. You see, I wanted to delegate this functionality to an external tool – we do this to extract the .7z archives, by the way. Something like calling an exe and specify the folder and the path to the .pak file to be created as command line parameters.

And that’s how the search started for .PAK file tools. It didn’t end well though: most of them were Windows applications, with no support for command line parameters. Don’t get me wrong, they are probably great tools, it’s just none was easy to interface with from MapTap.

Things turned for the better when finding the dirpack utility; apparently it was part of the QUtils package and then integrated with qcc. I already tried my hand with qcc and failed to convince it to produce a .pak file, so I took the bite, got the code for dirpack and started my attempts to compile it.

Let me tell you one thing: changing compiler specifications is bad. I bet that the code could produce executables, if you compiled it with an old enough compiler. But after an hour of hopelessly trying to put together a decent distribution of djgpp to compile it, I gave up on that approach and went on a different route: adapt the code to the current compilers.

My plan was to build it using CodeBlocks (it runs on MinGW), so the changes in the code went for that direction.

After fixing the obvious (renaming functions already available, finding the right headers to substitute some includes), I faced the first issue: ScanDir.

ScanDir is a function that returns the contents of a directory. You can even specify function pointers to do a bit of filtering and ordering. And it’s not provided for MinGW.

A look at the dirpack code however tells that these features were not used: the function was called with NULL for both function pointer parameters. This meant that even a crippled version of ScanDir would do the job, as long as it returns the complete list of files.

So, roll up the sleeves and start coding.

With that done, I was able to get the dirpack.exe compiled. Next step? Start testing. And see it failed.

Indeed, the .PAK file would not be created. Debugging the code, I found that file reading failed: the number of bytes returned by the read() call was not matching the size of the file. After horribly hacking the code, I finally got my first .PAK file. And surprise – surprise, when unpacking it the files were unusable.

Where could this error come from?

It turned out it was caused by the open() call. You see, a good time ago, all files were opened the same, regardless of their contents. There was no distinction made between text files and binary files. But then, as the open() function was ported to various platforms with their compilers, new rules were added to opening a file. Mainly, the O_BINARY flag, that specifies that for the file to be opened, all bytes from 0 to 255 are fair game.

With that flag in place, file operations ran fine. The .PAK files created could be extracted and the extracted content be used. Dirpack.exe rose from it’s ashes. MapTap support for Opposing Force is now possible.

I will release the code changes done for dirpack: who knows, somebody else might want to integrate it in their app.

Portal 2’s Perpetual Testing Initiative versus map sites

One question Shawn (the chief of modinformer.net) asked during the interview he conducted with Phillip and I (available here) was in the line of: “How do you think the Portal 2’s Perpetual Testing Initiative will affect map sites like PlanetPhillip.com?

I was caught off-guard. I didn’t have a proper look at what P2PI (short for Portal 2’s Perpetual Testing Initiative) means. My answer was more on the line of “not too worried, people looking for quality and for a well reviewed map will end up on PlanetPhillip.com eventually”.

Thing is, there is a threat element, low at the moment, but still, present.

You see, P2PPI is the name for the toolkit that Valve released for Portal 2. It makes it very easy to make maps (a good thing) and it makes it very easy to publish them through Steam Workshop.

And it is the Steam Workshop that is the threat, not the toolkit itself. So I opened up the browser, and looked at the list of maps available for Portal 2, looking for well articulate descriptions and, if any, reviews. Guess what: in my quick search (about 10 maps from the top favorites), none had a description longer than 200 words. As for reviews, let’s be serious, the community is more interested in playing rather than reviewing.

To me this is weird. I mean the tools for writing reviews (a darned editor to post a comment) is there, but nobody bothers writing them. This places PlanetPhillip.com in a privileged position, of still being top dog in terms of reviewing custom maps. For people who like to read a review first then decide to play the map, PlanetPhillip.com is still the place to go.

However, there is still a good number of people who just download maps based on just the rating it has, and they will favor the workshop.

Here’s the thing though: I think that those people would have downloaded maps from filefront ot whatever other file sharing site rather than from PlanetPhillip.com

So the threat, as I see it, does not consist in the Steam Workshop stealing visitors from PlanetPhillip.com, but rather in making it more difficult for Phillip to gain more users. There is always the danger that Steam will hire a couple of people to do reviews for those maps, and publish them alongside the original description of the map. Now THAT would spell bad news for Phillip.

Searching for a winner

About a month ago, Phillip decided to release MapTap. The application handled well enough of his .7z files (not all of them though, but we’re working on that) so the launch made sense. Now, in order to promote the application, Phillip organized a giveaway: one single MapTap user would get a small bundle of games.

The giveaway was something Phillip planned for the launch, so we had a little ‘automatic registration system’ in place: once the user installed the application, while running it, it would retrieve an ID (numerical) and a key (alphanumerical). When the giveaway would end, Phillip would post on his site the winning ID, and the user that had that ID would send him his key, to validate that he/she is indeed the winner. Phillip would then send the games.

It didn’t work out.

He selected (randomly) the first ID, posted it on his web site and waited for 3 days for the owned of that ID to mail him the key. Nothing happened. He then selected another ID, posted an update on his website, and then waited some more. Nothing happened.

Obviously, when searching for a winner something went wrong. But what? Let’s walk through what is my list of suspects:

  1. some MapTap users didn’t know they had an ID and key assigned to their MapTap
  2. some MapTap users didn’t know that there is a giveaway in place
  3. some MapTap users forgot about the giveaway
  4. some MapTap users  don’t track PlanetPhillip.com
  5. some MapTap users didn’t get the application from  PlanetPhillip.com, so they involuntarily joined the ranks of people at (2)

I don’t think that the first one is responsible though. I mean in his description of the giveaway, Phillip does mention where the ID and key can be read from.

It could be (2) (and by extension (5)), and that’s actually good news: word of mouth about MapTap is favorable. It means that the application has the potential of expanding the user base of the site, and Phillip would love to hear more about this.

It could be (4) (and (3), I mean if you forgot about the giveaway, you obviously don’t track the website), and this is a bit upsetting. Phillip wants the website to be the center of the user experience, while the app would just enhance the overall usability of PlanetPhillip.com users. They won’t need to manually install and uninstall the archives he posts, they just click a link and voila, the map is installed.

But if indeed (4) is the main culprit, then something needs to be done to make tracking easier.

One thing I suggested Phillip is to have some sort of configurable RSS feed. Just like you can filter the articles by selecting one or more tags, have something similar for RSS feeds. This way, people who believe that only map releases are important can subscribe only to that, and people who like polls and competitions can make a feed just for that.

If (2) and (5) are the main issue though, then Phillip is facing a nice opportunity to increase his user base. Question is, how should we do it? Again, some possible solutions:

  1. have a splash screen at startup, mentioning the website
  2. have a window pop up from time to time, hinting towards the website, or asking for feedback about either the app or the website
  3. have balloons in the system tray, mentioning the website, or pointing out links to new archives as they get added.
  4. or, more to this particular problem, have a pop up showing up on the winner’s computer saying “Congratulations, you won stuff!”

Obviously (2) and (4) are spamy. Avast does this when they update their virus signature database (well, they do something slightly worse with the default settings on, they’ll also play out a sound, a female voice saying that the virus database has been updated. Yuk!) and it feels bad. So we won’t do that. A splash screen (1) is however a good idea, and I’ll have that prepared.

What about balloons? This is where I have my doubts. Should we have them? And if so, what should be the message? Should we alert the user each time something happens on PlanetPhillip.com? Should we just ‘watch’ an RSS feed?

Now normally you would run a poll and take the decision according to that. But the people we’re targeting are not visiting the website, for a reason or another. They do use the app. And doing the poll through a pop up from MapTap is a no-go (see the spamy aspect of pop ups mentioned before).

I guess we’ll experiment. Start with the splash screen and move do a balloon thing watching a feed dedicated for MapTap. We’ll see how that goes.

Better interfaces to your data source

There is a reason why most Java programmers create classes that match the structure of their database tables. To get to that conclusion, let’s talk about a mistake I made when building MapTap.

So, initially MapTap was recording (in terms of data) just the name of the map from an archive. A simple file (should have been a database) listing the bsps would do the trick. But then we wanted to keep track on the date when each map was played and (this might come as a future feature) a rating.

Now, this data is being obtained by the main executable from a DLL. In order not to depend on the changes done for the DLL, we’re loading it with an explicit call of “LoadLibrary”. All nice and dandy.

Now, assuming that the first version of the DLL had this function:

int GetMapsForArchive (char * iszArchiveName, char *** oaszMaps);

and the main exe would do a GetProcAddr to find it in the DLL, and that you have two possible ways to extend the interface:

Method 1:

int GetMapsForArchive (char * iszArchiveName, char *** oaszMaps, char *** oaszPlayedDates, int ** oaRatings);

or Method 2:

struct MTFileData
    char * mapName;
    char * playedDate;
    int rating;

int GetMapsForArchive (char * iszArchiveName, struct MTFileData **oaFileData);

which one do you think it’s better?

You’re right, it is Method 2. There is one particular reason why the second one is better: less worries when the data obtained changes further.

If you remember what I mentioned earlier, in order to call the function from the DLL, we need to do a GetProcAddress. The function pointer returned by it must then be cast to the type of function we know of. This means that the actual type of function returned must match the signature of our local definition of the function pointer.

If we keep changing the signature of the function, we’ll also need to change the definition of the local function pointers. This is not only tedious, but outright dangerous: if you forget to do it, you end up with crashes.

By using data structures to encapsulate the format of the data, you’re delegating any interface changes from the function you call to the object you get. And that is much safer than having to change function signatures in several places, since you’ll only change the one header that defines that struct.

The Java guys got that right, and their constant use of databases keeps them from forgetting the lesson.