About Afr0

Programmer, artist, art-historian, journalist and fan of The Sims Online.

Rendering on Android

Flattr this!

Because I still haven’t really made any progress on rendering 3D with XNA, and just got a new phone (HTC Wildfire S), I’ve decided to branch out technology development to Android using OpenGL ES (Embeded Systems). It is a somewhat unlikely branch (because the likelyhood of Project Dollhouse being ported to Android is slim to none), but my hope is that I’ll be able to learn something that I can apply to XNA.

So far all I have is a spinning triangle using a texture from TSO, but I haven’t really gotten started on the programming yet because I spent quite a while setting up the development environment. I’ll update this post with pictures as soon as I have something interesting to show!

Afr0 Games is now also on Twitter! Come follow us!

Scaling done!

Flattr this!

The client now supports two resolutions:  800×600 and 1024×768 – without a border! I discovered that the graphics actually scale quite well, so I’m slightly puzzled as to why Maxis originally decided to put a border around the screen when in 1024 resolution. I suspect the answer might be that today, scaling is done on the GPU and as such is much less expensive – either way, the result is that the game now looks better than ever! 😀

Create A Sim

CAS in glorious, mouth-watering 1024×768!

Move to Github + working SPR#

Flattr this!

Today I moved the codebase from Assembla to Github, which will allow other developers to fork it and push new changes to the master branch! Hopefully the project will attract some attention from the Github community so that development will speed up.

In addition I finally completed SPR# decompression, so that all SPR# sprites now decompresses correctly! 😀

A decompressed wall sprite!

Progress update

Flattr this!

Since the last post, support has been added for more or less all the iff chunks in the game (with an exception for the BHAV and OBJD chunks).

I’ve added an analyzing feature to Iffinator, to aid me in rapidly supporting new chunks. The only major thing that is currently missing is decompression of SPR# sprites, which I’m currently working on. I’ll also eventually get back to 3D rendering, but mustering the motivation to do so is difficult. I really wish I had someone more experienced to help me with that. So if you’re reading this and you have experience with 3D rendering in C# and/or the XNA API, please do not hesitate to contact me!

As far as parsing SimAntics, I’ve implemented support for Sleep and Idle commands. More will be implemented over time, but progression will most likely speed up as more of the gameplay is implemented in the client.

Currently working on: SimAntics

Flattr this!

So, since the last update, I have been working on adding normal transformation and normal blending. I still haven’t been able to figure out correct bone placement and rotation, so for the time being I’ve put it aside to work on what is probably one of the most difficult aspects of this project;

Recreating the VM.

For those unfamiliar with the concept, VM is short for Virtual Machine, and the core of The Sims consists of a VM named Edith, which is constantly running the code inside objects (sims are also objects with their own programs for Edith to run).

Now, in TSO, the VM is less important because you only control one sim at a time, but you can still turn on free will to have your sim be autonomous.

The task of recreating Edith is so massive and all-encompassing  that it is hard to imagine where to start, but after quite a bit of research and cooperation with NIOTSO, I’ve decided to start by writing a basic parser for SimAntics (the language that Edith programs are written in) that translates it into a human-readable form. After that, the next step will be to design an interface so that SimAntics programs can easily call functions defined in the Project Dollhouse executable. If all goes well, after a certain amount of time I should have a VM built in C# that can load *.iff files in realtime and run their programs synchronously and display state changes on the screen. The Sims is basically just a graphical representation of such a machine.

How to compile and run the Project Dollhouse client

Flattr this!

Since so many people are asking about the client, seemingly completely unable to wait until it is released as a compiled binary for testing, I feel like I’m forced to make this blogpost explaining the neccessary steps.

Once you’ve grabbed the latest version of the repository, you should download and install Visual Studio C# Express Edition 2008 from here.

Next you’ll need to download and install XNA Gamestudio 3.1, then Lua for Windows, as well as Bass.NET. You can now open TSOClient.sln from trunk\TSOClient\ and compile the client. To compile an application in Visual Studio, simply press F5.

However, the client won’t run unless you have copied the Lua scripts to their appropriate directories. So for now, just press F5 to build the client. This will create a folder called bin in trunk\TSOClient\TSOClient\ with a folder called Debug inside.

The compiled client will be in the Debug folder. You need to create a folder called gamedata inside of Debug, and then copy all the three folders inside trunk\TSOClient\LuaScripts\ to this new folder. Lastly, copy NAudio.dll from trunk\TSOClient\TSOClient\ to trunk\TSOClient\TSOClient\bin\Debug\

When you’ve done this, you can open up Visual Studio again (if you closed it) and press F6 to compile and debug the client. Now the client should run without any errors, providing you have installed TSO on your PC.

Nearly there!

Flattr this!

After three, nearly four months of hard work, sweat, tears, frustration, research, teeth-grinding and hair-pulling, I’m nearly able to render a sim-body correctly;

If you can’t see that clearly, it is a maid’s body with the arms straight down and the legs tucked above the head. I’m not sure why the legs are like that, but undoubtedly it has something to do with the skeleton. Anyways, what’s important is that I’m nearly there!

Long live Project Dollhouse!

About the project’s timespan

Flattr this!

Today I was asked on Facebook when this project would be finished. I tried to explain that it doesn’t have any deadline, but the message didn’t seem to go through.
Therefore I’m writing this post to put things into perspective. One of the projects that inspired me to do this was Corsix-TH. It aims to rebuild Theme Hospital using nothing but the game’s original assets and open-source code. The difference between that project and this is that Theme Hospital is a much older game, and much less complicated to rebuild. Corsix-TH has been in development now since mid-2009, and it only just reached version 0.01.

Think about that for a minute. Between 0.01 and 1.00 you can at the least put 9 incremental updates. And if each update takes  6 months to make, that equals out to 4.5 years. Now as far as I know, there is currently only one developer working on Corsix-TH, and hopefully more developers will join as the project matures. That’s my hope for this project as well, but at least these figures should put things into perspective.

Progress…

Flattr this!

So far I’ve decided to concentrate on the skeleton, and the skeleton only, because everything else is depending on it.
This is the closest I’ve been able to come so far;

I’m still trying to figure out why the root bone is located away from the rest of the skeleton and why the upper arms are above the head, and the lower arms are way down. Other than that, everything seems to be correct.

Tension is rising…

Flattr this!

Have you ever tried hammering a nail into a wall using nothing but a rock, with both hands tied firmly behind your back?
That’s just about how it feels trying to do any 3D operations in XNA that are even remotely advanced. I have now tried for several months to set up skeletal animation, and am now in a situation where I’ve discovered that XNA even has a built-in function for parts of what I’ve been trying to do (transforming a set of vertices based on a bone’s location in 3D space), but that this is completely useless so long as I am unable to position the skeleton’s bones correctly in 3D space.
My current theory is that the bones’ locations (which are made up of a rotation and a translation) are stored as OpenGL-native coordinates. Vitaboy (the rendering system used by The Sims and The Sims Online) was originally written in Direct3D, but it isn’t unthinkable that it was ported to OpenGL for TSO. I have tried every possible matrix and vector based operation on the bones’ translations and rotations that I could possibly think of, but nothing seems to help.
I have been putting this off for what feels as long as possible, and I’m giving myself another week, but if I haven’t made any significant progress within that timeframe I’m switching to OpenGL.
I have already tried SFML (Simple Fast Multimedia Library – an engine that allows you to use OpenGL calls and combine them with calls to the engine), but it applies some kind of funky custom world matrix that fucks up animation and rendering.
The reason I’ve been putting this off for so long is that I know that rewriting the GUI-system in OpenGL is going to take time and be a learning process all of its own, but my current consensus is that it’ll take less time than trying to fight with XNA in order to make it do something it obviously wasn’t meant to do.
I am allowing comments on this post in the hope that I can get some bright ideas and/or support.

PHP and HTTP

Flattr this!

So, after a tip from Jayenkai over at [url=http://www.socoder.com]Socoder[/url], I decided to use a webserver after all, but I wrote a couple of PHP scripts to take care of finding the manifests.

One, called “patch.php”, is the most important. It sends the right manifest(s) to a client based on what version is passed to it.


<?php

/*The contents of this file are subject to the Mozilla Public License Version 1.1
(the "License"); you may not use this file except in compliance with the
License. You may obtain a copy of the License at http://www.mozilla.org/MPL/

Software distributed under the License is distributed on an "AS IS" basis,
WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for
the specific language governing rights and limitations under the License.

The Original Code is patch.php.

The Initial Developer of the Original Code is
Mats 'Afr0' Vederhus. All Rights Reserved.

Contributor(s): ______________________________________.
*/

include 'createzip.php';

if(isset($_GET['Version']))
{
	$ClientVersion = intval(htmlspecialchars($_GET['Version']));
	$NewManifest = "";

	if($handle = opendir(realpath('./patches/')))
	{
		//Find out if there are any manifests newer than the client's version.
		while (false !== ($entry = readdir($handle)))
		{
        		if(intval(str_replace('.manifest', '', $entry)) > $ClientVersion)
			{
				$NewManifest = realpath('./patches') . '/' . $entry . '.manifest';
				break;
			}
		}

		if(empty($NewManifest) || strcmp($NewManifest, realpath('./patches')) == 0)
		{
			header('Content-Description: No New Manifest');
			header('Content-Type: text/html; charset=utf-8');
			echo('<html><body><p>No new manifest!</p></body></html>');
		}
		else
		{
			//Open the new manifest and check if it has a child, which means an
			//incremental update needs to take place...
			$FileHandle = fopen($NewManifest, 'r');

			$Parent = fgets($FileHandle);
			$CurrentChild = fgets($FileHandle);
			$Path = realpath('./patches');

			$Parent = str_replace('Parent=', '', $Parent);
			$Parent = str_replace('"', '', $Parent);
			$Parent = trim($Parent);
			$CurrentChild = str_replace('Child=', '', $CurrentChild);
			$CurrentChild = str_replace('"', '', $CurrentChild);
			$CurrentChild = trim($CurrentChild);

			fclose($FileHandle);
			
			//This is a compromise. Instead of checking for all childs, just check for the first one,
			//and let the client handle the other children (if any).
			if(empty($CurrentChild) !== true && empty($Parent) !== true)
			{
				$Parent = $Path . '/' . $Parent;
				$CurrentChild = $Path . '/' . $CurrentChild;

				$FilesToZip = array($CurrentChild, $NewManifest);
				$ZipName = './' . strval(rand()) . '.zip';
				create_zip($FilesToZip, $ZipName);
				
				//Client should check the Content-Description header to figure out what to expect.
				header('Content-Description: Zipped File Transfer');
    				header('Content-Type: application/octet-stream');
    				header('Content-Disposition: attachment; filename='.basename($ZipName));
    				header('Content-Transfer-Encoding: binary');
    				header('Expires: 0');
    				header('Cache-Control: must-revalidate, post-check=0, pre-check=0');
    				header('Pragma: public');
    				header('Content-Length: ' . filesize($ZipName));
    				ob_clean();
    				flush();
    				readfile($ZipName);

				unlink($ZipName);
				exit();
			}
			//There was a child, but no parent.
			else if(empty($CurrentChild) !== true && empty($Parent) == true)
			{
				$CurrentChild = $Path . '/' . $CurrentChild;

				$FilesToZip = array($CurrentChild, $NewManifest);
				$ZipName = './' . strval(rand()) . '.zip';
				create_zip($FilesToZip, $ZipName);
				
				//Client should check the Content-Description header to figure out what to expect.
				header('Content-Description: Zipped File Transfer');
    				header('Content-Type: application/octet-stream');
    				header('Content-Disposition: attachment; filename='.basename($ZipName));
    				header('Content-Transfer-Encoding: binary');
    				header('Expires: 0');
    				header('Cache-Control: must-revalidate, post-check=0, pre-check=0');
    				header('Pragma: public');
    				header('Content-Length: ' . filesize($ZipName));
    				ob_clean();
    				flush();
    				readfile($ZipName);

				unlink($ZipName);
				exit();
			}
			//There was a parent, but no child.
			else if(empty($Parent) !== true && empty($CurrentChild) == true)
			{
				$Parent = $Path . '/' . $Parent;

				$FilesToZip = array($Parent, $NewManifest);
				$ZipName = './' . strval(rand()) . '.zip';
				create_zip($FilesToZip, $ZipName);
				
				//Client should check the Content-Description header to figure out what to expect.
				header('Content-Description: Zipped File Transfer');
    				header('Content-Type: application/octet-stream');
    				header('Content-Disposition: attachment; filename='.basename($ZipName));
    				header('Content-Transfer-Encoding: binary');
    				header('Expires: 0');
    				header('Cache-Control: must-revalidate, post-check=0, pre-check=0');
    				header('Pragma: public');
    				header('Content-Length: ' . filesize($ZipName));
    				ob_clean();
    				flush();
    				readfile($ZipName);

				unlink($ZipName);
				exit();
			}
			//There was neither a parent nor a child. This should really never occur.
			else
			{
				header('Content-Description: File Transfer');
    				header('Content-Type: application/octet-stream');
    				header('Content-Disposition: attachment; filename='.basename($NewManifest));
    				header('Content-Transfer-Encoding: binary');
    				header('Expires: 0');
    				header('Cache-Control: must-revalidate, post-check=0, pre-check=0');
    				header('Pragma: public');
    				header('Content-Length: ' . filesize($NewManifest));
    				ob_clean();
    				flush();
    				readfile($NewManifest);

				exit();
			}
		}
	}
}
else
{
	header('Content-Description: Invalid Request');
	header('Content-Type: text/html; charset=utf-8');
	echo('<html><body><p>Invalid request!</p></body></html>');
}
?>

This code depends on another script I found while rummaging through Google:

<?php
//From: http://davidwalsh.name/create-zip-php
function create_zip($files = array(),$destination = '',$overwrite = false) {
  //if the zip file already exists and overwrite is false, return false
  if(file_exists($destination) && !$overwrite) { return false; }
  //vars
  $valid_files = array();
  //if files were passed in...
  if(is_array($files)) {
    //cycle through each file
    foreach($files as $file) {
      //make sure the file exists
      if(file_exists($file)) {
        $valid_files[] = $file;
      }
    }
  }
  //if we have good files...
  if(count($valid_files)) {
    //create the archive
    $zip = new ZipArchive();
    if($zip->open($destination,$overwrite ? ZIPARCHIVE::OVERWRITE : ZIPARCHIVE::CREATE) !== true) {
      return false;
    }
    //add the files
    foreach($valid_files as $file) {
      $zip->addFile($file,$file);
    }
    //debug
    //echo 'The zip archive contains ',$zip->numFiles,' files with a status of ',$zip->status;
    
    //close the zip -- done!
    $zip->close();
    
    //check to make sure the file exists
    return file_exists($destination);
  }
  else
  {
    return false;
  }
}
?>

The ‘patch.php’ script can be utilized by clients as such;

http://www.webserver.com/patch.php?Version=ClientVersion

Lastly there’s the ‘getmanifest.php’ script which simply retrieves a specific manifest;

<?php
/*The contents of this file are subject to the Mozilla Public License Version 1.1
(the "License"); you may not use this file except in compliance with the
License. You may obtain a copy of the License at http://www.mozilla.org/MPL/

Software distributed under the License is distributed on an "AS IS" basis,
WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for
the specific language governing rights and limitations under the License.

The Original Code is getmanifest.php.

The Initial Developer of the Original Code is
Mats 'Afr0' Vederhus. All Rights Reserved.

Contributor(s): ______________________________________.
*/

if(isset($_GET['Manifest']))
{
	$RequestedManifest = htmlspecialchars($_GET['Manifest']);
	$FoundManifest = "";

	if($handle = opendir('./patches/'))
	{
		//Find the requested manifest...
		while (false !== ($entry = readdir($handle)))
		{
        		if($entry === $RequestedManifest)
			{
				$FoundManifest = './patches/' . $entry;
				break;
			}
		}

		if(empty($FoundManifest) || strcmp($FoundManifest, './patches') == 0)
		{
			header('Content-Description: Invalid Request');
			header('Content-Type: text/html; charset=utf-8');
			echo('<html><body><p>No manifest by that name exists!</p></body></html>');
		}
		else
		{
			header('Content-Description: File Transfer');
    			header('Content-Type: application/octet-stream');
    			header('Content-Disposition: attachment; filename='.basename($FoundManifest));
    			header('Content-Transfer-Encoding: binary');
    			header('Expires: 0');
    			header('Cache-Control: must-revalidate, post-check=0, pre-check=0');
    			header('Pragma: public');
    			header('Content-Length: ' . filesize($FoundManifest));
    			ob_clean();
    			flush();
    			readfile($FoundManifest);

			exit();
		}
	}
}
else
{
	header('Content-Description: Invalid Request');
	header('Content-Type: text/html; charset=utf-8');
	echo('<html><body><p>Invalid request!</p></body></html>');
}
?>

It can be utilized as such;

http://www.webserver.com/getmanifest.php?Manifest=ManifestVersion.manifest

I also created a tool for generating these manifests. The important thing, however, is that the two first lines of a manifest are stored as plain ASCII strings so they can be read by ‘patch.php’. Aside from that, manifests can look however you want them to.
Here’s a sample manifest generated by Manifestation;

Child=””
NumFiles=48
&DLLs\Aries.dll MD5: ?5??????
I?[/?
*DLLs\authlogin.dll MD5: ?????
(HitListsTemp.dat MD5: ~l?{??JL,?2?9?
.DLLs\HitMp3DecodeD.dll MD5: ????psCX?p’??2%6
&DLLs\ijl10.dll MD5: ?r?q.GcP??
1DLLs\InternetServiceD.dll MD5: ? y?fU?|[?ro???
8DLLs\TSONetServiceSimClientD.dll MD5: !???m??>`

The added MD5s after filenames are checksums that can be used by clients to verify file integrity.
If you’re looking for a general-purpose way to patch your game(s), I hope I’ve given you some food for thought!

FTP vs. HTTP

Flattr this!

Right. I just tried getting the FTP server I’ve been working on to work with FileZilla for the umpteenth time. It still doesn’t work, no matter what I do.

Unfortunately, the FTP-protocol is extremely badly documented (aside from the documents available from IETF, which are extremely terse, contains unneeded bloat and are generally just downright bad). I’ve been able to make my way up to and including the LIST and/or MLSD command (the MLSD being a newer extension command), but when trying to transfer the data containing the directory listing for the root directory, FileZilla never acknowledges that the data was received! Instead it times out and decides that there was a problem receiving the directory listing.

So, I’ve been thinking, would it make sense to just switch to HTTP for transferring patches to clients? It is starting to look extremely tempting. Arguably, I could code my way around the FTP-protocol by introducing non-standard protocol “extensions”, but that would defeat the purpose of using FTP in the first place (namely that it’s a pre-designed filetransfer protocol).

I don’t know if I’ll have the time tomorrow (I should study), but one of these days I’m going to install a copy of Apache on my box and see about making my patch client communicate with it.

So far I’ve been thinking of two possible solutions for a directory structure on the server:

  • Dumping a manifest  and all files into the root folder. This would have the downside of making it near-impossible to do incremental updates, unless the manifest contained information about which files belonged to which update.
  • Have a manifest in the root folder that points to different versions contained in separate directories. This seems like the best way to go, as it would enable incremental updates without making the manifest needlessly complex.

Once I’ve been able to write some basic HTTP code for the patch client, I’ll make another post with pictures of the client to inform of the progress.

3D

Flattr this!

After a lot of hair pulling, teeth grinding, swearing and performing sacrificial rituals to Microsoft in my back garden, I finally figured out how to render meshes correctly on top of textures (with some help from the Gamedev.net forums).

The "Create A Sim" screen

3D head being rendered in the client

As you can see, there’s still stuff to figure out, such as correct rotation and placement in 2D space. There’s also the issue of selecting the right mesh when the corresponding button is pressed, which is currently a little off. But the rendering is in place, and that’s the important thing for now.

New website up!

Flattr this!

Finally, Project Dollhouse has its own website. I’ve been putting it off for a long time due to working on the code, but I figured it was time to make it happen. In this section I will be blogging about any news related to the project.

Please forgive if things aren’t looking “up to scratch” at the moment, as I am still tweaking the setup to make the website just the way I want it.