Finally back and gapbuffers

Flattr this!

Finally, I’m back!

I’ve been working on this project ever since the time of the last post. Since then I’ve rewritten pretty much the entire codebase from the ground up. There are many things I could write about – a whole host of them, in fact. 

But I’m going to start by writing about what I’m currently working on: a textbox. It is probably the hardest UI component to write.

I tried doing it once, and it didn’t work. I ended up writing over 1000 lines of code on my own. It worked – sorta – you could type and scroll text up, down and sideways. You could even delete text, and multiple lines of text were supported. But things got rather hairy when I was going to delete characters from within a sentence. Bugs started popping up seemingly out of nowhere, and after postponing it for the longest time, I decided that the best thing to do would be to simply do a rewrite.

After presenting my problem to the Gamedev discord channel, they suggested that a gap buffer would be a reasonable solution to my woes. And, being the lazy programmer that I am, I went in search of an existing implementation. I found it, like many times before, at Codeproject. A gap buffer, simply put, is a collection of items with a gap in it that can be moved in order to facilitate fast insert, removal and lookup.

I also decided to move the rendering part of the code into its own class, a TextRenderer. This was a sound decision – the code is now a  lot easier to debug. Thankfully though, using a GapBuffer has resulted in much more robust code that doesn’t need as much debugging.

Here’s all the code I ripped out of the UITextEditor class:

using System.Collections.Generic;
using Microsoft.Xna.Framework;
using Microsoft.Xna.Framework.Graphics;

namespace Gonzo
{
    /// <summary>
    /// Represents a line of renderable text.
    /// </summary>
    internal class RenderableText2
    {
        public string Text = "";
        public Vector2 Position;
        public bool Visible = true;
    }

    /// <summary>
    /// Represents a renderable character.
    /// </summary>
    internal class RenderableCharacter
    {
        public string Char = "";
        public Vector2 Position;
        public bool Visible = true;
    }

    /// <summary>
    /// Responsible for rendering text for UITextEdit.
    /// </summary>
    public class TextRenderer
    {
        /// <summary>
        /// Is this renderer for a single or multi line textbox?
        /// </summary>
        bool m_MultiLine = false;

        private Color m_TextColor;
        private SpriteFont m_Font;
        private Vector2 m_CurrentTextPosition = new Vector2(); //Position of the character currently being added.
        int m_VisibilityIndex = 0; //The index for the line beyond which nothing is rendered.
        private float m_LineHeight;
        private Vector2 m_TextboxPosition;
        private Vector2 m_TextboxSize;
        private int m_ScrollFactor = 0;
        private List<RenderableText2> m_RenderableLines = new List<RenderableText2>();
        private GapBuffer<RenderableCharacter> m_CurrentLine = new GapBuffer<RenderableCharacter>();

        /// <summary>
        /// The contents of the GapBuffer containing the current line.
        /// </summary>
        /// <returns>The contents of the GapBuffer containing the current line.</returns>
        private string GetCurrentLine()
        {
            string Output = "";

            foreach (RenderableCharacter Char in m_CurrentLine)
                Output += Char.Char;

            return Output;
        }

        /// <summary>
        /// Constructs a new TextRenderer instance.
        /// </summary>
        /// <param name="TextboxPosition">Position of textbox using this renderer.</param>
        /// <param name="TextboxSize">Size of textbox using this renderer.</param>
        /// <param name="ScrollFactor">Scrolling factor of textbox using this renderer.</param>
        /// <param name="LineHeight">The height of a line of text.</param>
        /// <param name="Font">The font used to render the text.</param>
        public TextRenderer(bool MultiLine, Vector2 TextboxPosition, Vector2 TextboxSize, int ScrollFactor, 
            float LineHeight, SpriteFont Font, Color TxtColor)
        {
            m_MultiLine = MultiLine;
            m_TextboxPosition = TextboxPosition;
            m_CurrentTextPosition = m_TextboxPosition;
            m_TextboxSize = TextboxSize;
            m_ScrollFactor = ScrollFactor;
            m_LineHeight = LineHeight;
            m_Font = Font;
            m_TextColor = TxtColor;
        }

        /// <summary>
        /// Scrolls the current line of text to the right.
        /// </summary>
        public void ScrollTextRight()
        {
            for (int i = 0; i < m_CurrentLine.Count; i++)
            {
                m_CurrentLine[i].Position.X += m_ScrollFactor;

                if (m_CurrentLine[i].Position.X > (m_TextboxPosition.X + m_TextboxSize.X))
                    m_CurrentLine[i].Visible = false;

                if (m_CurrentLine[i].Position.X > m_TextboxPosition.X && 
                    m_RenderableLines[i].Position.X < (m_TextboxPosition.X + m_TextboxSize.X))
                    m_CurrentLine[i].Visible = true;
            }
        }

        /// <summary>
        /// Scrolls the current line of text to the left.
        /// </summary>
        /// <param name="Index">An optional index. If it's set, only the text from the index onwards will scroll.</param>
        public void ScrollTextLeft(int Index = 0)
        {
            for (int i = Index; i < m_CurrentLine.Count; i++)
            {
                m_CurrentLine[i].Position.X -= m_ScrollFactor;

                if (m_CurrentLine[i].Position.X < (m_TextboxPosition.X + m_TextboxSize.X))
                    m_CurrentLine[i].Visible = true;

                if (m_CurrentLine[i].Position.X > (m_TextboxPosition.X + m_TextboxSize.X) &&
                    m_CurrentLine[i].Position.X < m_TextboxPosition.X)
                    m_CurrentLine[i].Visible = false;
            }
        }

        /// <summary>
        /// Scrolls all text up.
        /// </summary>
        /*public void ScrollTextUp()
        {
            foreach (RenderableText2 Txt in m_RenderableLines)
                Txt.Position.Y -= m_LineHeight;

            m_RenderableLines[m_VisibilityIndex].Visible = false;
            m_VisibilityIndex++;
        }*/

        /// <summary>
        /// Removes a character of renderable text from the input at the specified index.
        /// </summary>
        /// <param name="Index">The index at which to remove a renderable character.</param>
        public void RemoveAt(int Index)
        {
            if (Index > 0 && m_CurrentLine.Count >= 1)
            {
                m_CurrentTextPosition.X -= m_Font.MeasureString(m_CurrentLine[Index].Char).X;

                RenderableCharacter RenderChar = m_CurrentLine[Index];
                RenderChar.Char = "";

                m_CurrentLine.RemoveAt(Index);
                m_CurrentLine.Insert(Index, RenderChar);
            }
        }

        public void Insert(int Index, string Char)
        {
            if (m_MultiLine)
            {
                if (m_CurrentTextPosition.X < m_TextboxSize.X)
                {
                    m_CurrentTextPosition.X += m_Font.MeasureString(Char).X;
                    RenderableCharacter RenderChar = new RenderableCharacter();
                    RenderChar.Position = m_CurrentTextPosition;
                    RenderChar.Visible = true;
                    RenderChar.Char = Char;

                    m_CurrentLine.Insert(Index, RenderChar);
                }
                else
                {
                    RenderableText2 Line = new RenderableText2();
                    Line.Text = GetCurrentLine();
                    Line.Position = new Vector2(m_TextboxPosition.X + 2, m_CurrentTextPosition.Y);
                    Line.Visible = true;

                    m_RenderableLines.Add(Line);

                    m_CurrentTextPosition.Y += m_Font.LineSpacing;
                    m_CurrentLine.Clear();

                    m_CurrentTextPosition.X = m_TextboxPosition.X;
                    RenderableCharacter RenderChar = new RenderableCharacter();
                    RenderChar.Position = m_CurrentTextPosition;
                    RenderChar.Visible = true;
                    RenderChar.Char = Char;

                    m_CurrentLine.Insert(0, RenderChar);
                }
            }
            else
            {
                if (m_CurrentTextPosition.X < m_TextboxSize.X)
                    m_CurrentTextPosition.X += m_Font.MeasureString(Char).X;
                else
                    ScrollTextLeft();

                RenderableCharacter RenderChar = new RenderableCharacter();
                RenderChar.Position = m_CurrentTextPosition;
                RenderChar.Char = Char;

                m_CurrentLine.Insert(Index, RenderChar);
            }
        }

        /// <summary>
        /// Renders the text stored by this TextRenderer instance.
        /// </summary>
        /// <param name="SBatch">A SpriteBatch instance for drawing the text.</param>
        /// <param name="Depth">The depth at which to draw the text.</param>
        public void DrawText(SpriteBatch SBatch, float Depth)
        {
            if (m_MultiLine)
            {
                foreach (RenderableCharacter Txt in m_CurrentLine)
                {
                    if (Txt.Visible)
                    {
                        SBatch.DrawString(m_Font, Txt.Char, new Vector2(Txt.Position.X, Txt.Position.Y),
                            m_TextColor, 0.0f, new Vector2(0.0f, 0.0f), 1.0f, SpriteEffects.None, Depth + 0.1f);
                    }
                }

                foreach (RenderableText2 Txt in m_RenderableLines)
                {
                    if (Txt.Visible)
                    {
                        SBatch.DrawString(m_Font, Txt.Text, new Vector2(Txt.Position.X, Txt.Position.Y),
                            m_TextColor, 0.0f, new Vector2(0.0f, 0.0f), 1.0f, SpriteEffects.None, Depth + 0.1f);
                    }
                }
            }
            else
            {
                foreach (RenderableCharacter Txt in m_CurrentLine)
                {
                    if (Txt.Visible)
                    {
                        SBatch.DrawString(m_Font, Txt.Char, new Vector2(Txt.Position.X, Txt.Position.Y),
                            m_TextColor, 0.0f, new Vector2(0.0f, 0.0f), 1.0f, SpriteEffects.None, Depth + 0.1f);
                    }
                }
            }
        }

        private bool m_MaxScrollup = false;

        /// <summary>
        /// Can the text be scrolled up any further?
        /// </summary>
        public bool MaxScrollup
        {
            get { return m_MaxScrollup; }
        }


        private bool m_MaxScrolldown = false;

        /// <summary>
        /// Can the text be scrolled down any further?
        /// </summary>
        public bool MaxScrolldown
        {
            get { return m_MaxScrolldown; }
        }

        /// <summary>
        /// Performs the memory movement of scrolling text down.
        /// </summary>
        public void ScrollTextDown()
        {
            for (int i = 0; i < m_RenderableLines.Count; i++)
            {
                if (m_RenderableLines[m_RenderableLines.Count - 1].Visible == true)
                    m_MaxScrolldown = true;

                if (!MaxScrolldown)
                {
                    m_RenderableLines[i].Position.Y -= m_Font.LineSpacing;

                    if (m_RenderableLines[i].Position.Y <= m_TextboxPosition.Y && m_RenderableLines[i].Visible)
                        m_RenderableLines[i].Visible = false;

                    if (MaxScrollup == true)
                        m_MaxScrollup = false;

                    m_VisibilityIndex++;
                }
            }
        }

        /// <summary>
        /// Performs the memory movement of scrolling text up.
        /// </summary>
        public void ScrollTextUp()
        {
            for (int i = 0; i < m_RenderableLines.Count; i++)
            {
                if (m_RenderableLines[0].Visible == true)
                    m_MaxScrollup = true;

                if (!MaxScrollup)
                {
                    m_RenderableLines[i].Position.Y += m_Font.LineSpacing;

                    if (m_RenderableLines[i].Position.Y >= ((m_TextboxPosition.Y + m_TextboxSize.Y) - m_Font.LineSpacing) && m_RenderableLines[i].Visible)
                        m_RenderableLines[i].Visible = false;

                    if (MaxScrolldown == true)
                        m_MaxScrolldown = false;

                    m_VisibilityIndex--;
                }
            }
        }
    }
}

I decided to use one GapBuffer for the line of text that is currently being edited. This turned out to make the code I had a lot easier to work with.

What remains right now is the ability to delete text when it has scrolled past the borders of the text editor. And, eventually, the ability to select and copy/paste text.

Authentication and encryption

Flattr this!

Lately I’ve been working on safe authentication and encryption for Project Dollhouse’s network library, GonzoNet. After a lot of deliberation, the choice fell on Elliptic Curve Diffie Hellman (ECDH) for authentication and AES for encryption.

ECDH is relatively simple to implement, and is safe enough for the purposes of a game. It also makes it simple to establish encryption parameters net ieded for further encryption of a protocol.

In other news, we’re making great progress on the Simantics VM, so that the only things remaining are network related features.

For an example of how GonzoNet implements ECDH, and how you can use it in your own projects, look here!

KISS

Flattr this!

KISS is a well known acronym in design circuits. According to Wikipedia, it was first noted by the U.S Navy in the sixties.

When I was going for my second attempt at creating a fully fledged auto updating framework, this acronym flashed before my eyes before I’d written as much as a single line of code. The last one I did was over designed, and I could never get it to work the way I wanted it to.

The main way in which it was over designed was that it kept a record of previous software versions/patches. This was stupid. It required too much bookkeeping, for absolutely no gain. By the time new patches are deployed, any information in a previous patch is outdated anyway.

Therefore, I took it upon myself to do it, this time, the Right Way (to paraphrase John Carmack.)

The first vital decision I made was to put the actual downloading code into a library, so that it could be separated from the application itself and be reused. Obviously, I named the library KISS.

The second vital decision I made was to ignore any notion of previous versions/patches. This resulted in the following:

using System;
using System.Collections.Generic;
using System.Text;
using System.Net;
using System.Net.Security;
using System.IO;
using System.Threading;
using System.Security.Cryptography.X509Certificates;

namespace KISS
{
    public delegate void FetchedManifestDelegate(ManifestFile Manifest);
    public delegate void FetchedFileDelegate(MemoryStream FileStream);
    public delegate void DownloadTickDelegate(RequestState State);

    /// <summary>
    /// A requester requests and fetches files from a webserver, and calls
    /// events when the requests are complete.
    /// </summary>
    public class Requester
    {
        //This is implementation specific. Personally, I'm ignoring security to avoid headaches.
        public static bool ACCEPT_ALL_CERTIFICATES = true;

        public event FetchedManifestDelegate OnFetchedManifest;
        private string m_ManifestAddress = "";

        /// <summary>
        /// Contains information about a download in progress,
        /// such as: percent complete and KB/sec.
        /// </summary>
        public event DownloadTickDelegate OnTick;

        /// <summary>
        /// Called when a file was fetched!
        /// </summary>
        public event FetchedFileDelegate OnFetchedFile;

        private bool m_HasFetchedManifest = false;

        public Requester(string ManifestAddress)
        {
            m_ManifestAddress = ManifestAddress;
        }

        public void Initialize()
        {
            try
            {
                WebRequest Request = WebRequest.Create(m_ManifestAddress);
                RequestState ReqState = new RequestState();

                ReqState.Request = Request;
                ReqState.TransferStart = DateTime.Now;
                Request.BeginGetResponse(new AsyncCallback(GotInitialResponse), ReqState);
                ServicePointManager.ServerCertificateValidationCallback = new RemoteCertificateValidationCallback(AcceptAllCertifications);
            }
            catch (Exception E)
            {
                Logger.Log("Exception in Requester.Initialize:\n" + E.ToString(), LogLevel.error);
            }
        }

        /// <summary>
        /// Starts fetching a file, and notifies the OnFetchedFile event
        /// when done.
        /// </summary>
        public void FetchFile(string URL)
        {
            WebRequest Request = WebRequest.Create(URL);
            Request.Method = "GET";
            RequestState ReqState = new RequestState();

            ReqState.Request = Request;
            ReqState.TransferStart = DateTime.Now;
            Request.BeginGetResponse(new AsyncCallback(GotInitialResponse), ReqState);
            ServicePointManager.ServerCertificateValidationCallback = new RemoteCertificateValidationCallback(AcceptAllCertifications);
        }

        private void GotInitialResponse(IAsyncResult AResult)
        {
            RequestState ReqState = (RequestState)AResult.AsyncState;
            ReqState.Response = ReqState.Request.EndGetResponse(AResult);
            ReqState.ContentType = ReqState.Response.ContentType;
            ReqState.ContentLength = (int)ReqState.Response.ContentLength;

            Stream ResponseStream = ReqState.Response.GetResponseStream();
            ReqState.ResponseStream = ResponseStream;
            ReqState.RequestBuffer = new byte[ReqState.ContentLength];
            ResponseStream.BeginRead(ReqState.RequestBuffer, 0, (int)ReqState.Response.ContentLength,
                new AsyncCallback(ReadCallback), ReqState);
        }

        private void ReadCallback(IAsyncResult AResult)
        {
            RequestState ReqState = ((RequestState)(AResult.AsyncState));

            Stream ResponseStream = ReqState.ResponseStream;

            // Get results of read operation
            int BytesRead = ResponseStream.EndRead(AResult);

            // Got some data, need to read more
            if (BytesRead > 0)
            {
                // Report some progress, including total # bytes read, % complete, and transfer rate
                ReqState.BytesRead += BytesRead;
                ReqState.PctComplete = ((double)ReqState.BytesRead / (double)ReqState.ContentLength) * 100.0f;

                // Note: bytesRead/totalMS is in bytes/ms. Convert to kb/sec.
                TimeSpan totalTime = DateTime.Now - ReqState.TransferStart;
                ReqState.KBPerSec = (ReqState.BytesRead * 1000.0f) / (totalTime.TotalMilliseconds * 1024.0f);

                OnTick(ReqState);

                // Kick off another read
                IAsyncResult ar = ResponseStream.BeginRead(ReqState.RequestBuffer, ReqState.BytesRead,
                    (ReqState.RequestBuffer.Length - ReqState.BytesRead), new AsyncCallback(ReadCallback), ReqState);
                return;
            }

            // EndRead returned 0, so no more data to be read
            else
            {
                ResponseStream.Close();
                ReqState.Response.Close();
                ReqState.ResponseStream.Close();
                OnFinishedFile(new MemoryStream(ReqState.RequestBuffer));
            }
        }

        /// <summary>
        /// Finished downloading a file!
        /// </summary>
        /// <param name="FileStr">The stream of the file that was downloaded.</param>
        private void OnFinishedFile(MemoryStream FileStr)
        {
            if (!m_HasFetchedManifest)
            {
                m_HasFetchedManifest = true;
                OnFetchedManifest(new ManifestFile(FileStr));
            }
            else
            {
                OnFetchedFile(FileStr);
            }
        }

        private bool AcceptAllCertifications(object sender, X509Certificate certification, X509Chain chain, SslPolicyErrors sslPolicyErrors)
        {
            return ACCEPT_ALL_CERTIFICATES;
        }
    }
}

This is just sexy. It is short, to the point, easy to maintain, flexible and elegant. I will eventually port this to C++.

To explain, the KISS framework does most of its work based on a local and remote manifest file, which can be generated by a tool called Manifestation. A manifest file simply contains the version of a patch (or the client’s current version, if the manifest is a local one), the number of files contained in the manifest, and then a listing of the files that make up the patch. A file is defined as:

  • Address – the local address of the file, starting from the root folder of the client.
  • Hash – Hash of a file, to determine if the file has changed (and thus, if it needs to be downloaded).
  • URL – The URL of the file.

The actual code is equally simple:

using System;
using System.Collections.Generic;
using System.Text;
using System.IO;

namespace KISS
{
    /// <summary>
    /// A manifest file is a file that has a version and references a bunch of patch files.
    /// </summary>
    public class ManifestFile
    {
        public string Version = "";
        public List<PatchFile> PatchFiles = new List<PatchFile>();

        public ManifestFile(string Path, string Version, List<PatchFile> PatchFiles)
        {
            bool HasURLs = false;
            BinaryWriter Writer = new BinaryWriter(File.Create(Path));
            Writer.Write((string)Version);

            if (PatchFiles[0].URL != "")
                HasURLs = true;

            foreach (PatchFile PFile in PatchFiles)
            {
                if (!HasURLs)
                    Writer.Write((string)PFile.Address + "," + PFile.FileHash);
                else
                    Writer.Write((string)PFile.Address + "," + PFile.FileHash + PFile.URL);
            }

            Writer.Flush();
            Writer.Close();
        }

        /// <summary>
        /// Creates a ManifestFile instance from a downloaded stream.
        /// </summary>
        /// <param name="ManifestStream"></param>
        public ManifestFile(Stream ManifestStream)
        {
            BinaryReader Reader = new BinaryReader(ManifestStream);
            Reader.BaseStream.Position = 0; //IMPORTANT!

            Version = Reader.ReadString();
            int NumFiles = Reader.ReadInt32();

            for(int i = 0; i < NumFiles; i++)
            {
                string PatchFileStr = Reader.ReadString();
                string[] SplitPatchFileStr = PatchFileStr.Split(",".ToCharArray());

                PatchFiles.Add(new PatchFile()
                {
                    Address = SplitPatchFileStr[0],
                    FileHash = SplitPatchFileStr[1],
                    URL = SplitPatchFileStr[2]
                });
            }

            Reader.Close();
        }
    }
}

This has but one disadvantage: A hash is stored as text, so that comparing them isn’t particularly fast, but this can obviously be changed to suit your need if you decide to use the framework. My plan is to eventually port the framework to C++; for now, it is available as C# code at Github. The license is Mozilla 2.0 – my personal favorite. If you change anything, you have to share it. Other than that, use as you please!

Going live

Flattr this!

As I’m writing this, I’m at Oslo Airport Gardermoen, waiting for a plane to take me back to Bodø.

Technically speaking, Kon Tiki was considered done a couple of days ago. I have procured a server at Accuwebhosting, and the Loginserver is currently running.

However, I am unable to log in with the client, most likely because of an issue known as NAT traversal. What this means is that packets sent from a client behind a router need to find their way to the server (which is also typically behind a router) and vice versa.

It is possible to do this manually by forwarding ports in one’s router, but I am trying to avoid this by programmatically forwarding ports. This means more code and a few more days of waiting, but everything good shall come to those who wait!

Back on track!

Flattr this!

So, after a prolonged setback, I finally managed to compile the client using XNA 4. That means that development is now back on track, and I’m going to be debugging some network issues with the client before attempting to pull everything together to a release! 😀
The neat thing about this is that we probably (hopefully) have a working Monogame client as well, but I haven’t dared to try to run it yet, due to Monogame’s inherently unstable nature.

Windows 8 rant

Flattr this!

Seems like the Kon Tiki milestone might have to be postponed till the end of August.

I’m currently trying to get Visual Studio 2008 installed on my laptop, and it is proving to be an incredibly frustrating experience. This leads me to regret that we haven’t ported the entire codebase to Monogame by now, as well as regretting the decision to install Win8 on the laptop.

It started with error 1935, which apparently is a catch all errorcode for unspecified problems during installation. However, when you get an hresult of 0x80073712, it means that your component store is corrupted.

So far, so bad. If only this problem could be fixed. But it turns out, it can’t. Not readily, anyway. I’ve tried running DISM with /scanhealth and /restorehealth – no luck (aside from confirming what I already knew – component store corruption).

I then tried running the Windows Update Diagnostic tool, which fixed something… but VS 2008 still won’t install. It fails at pretty much exactly the same place, so I haven’t even bothered checking the logs to see if the errorcode changed.

I’ve also run CCleaner to fix any potential registry issues, still no go. If anyone have had any similar experiences, please let me know below!

Argh, Microsoft!

Packet dimensions

Flattr this!

Development has been paused for a while, mostly due to everyone being busy doing other stuff. However, now we are getting back into the swing of things.

Yesterday, I discovered that for some reason, I had decided that it was a good idea to encode a packet’s length in a single byte when I first implemented the game’s networking code.

Clearly, it isn’t. Unless you want to start packing bits to save space (which quite frankly isn’t worth the time and effort), a byte can only represent numbers up to 255. That means packets could only be 255 bytes long. This isn’t very effective, or even practical.

Therefore I decided to use an unsigned short (2 bytes) to encode a packet’s length, and another to encode a packet’s decrypted length (if it is encrypted). This means packets can now be up to 65,025 bytes in length. If I find myself needing to send packets any larger than this, I know I need to redesign the protocol.

Of course, doing this made sure that decoding packets received from the client fucked up. There’s always something. But at least development is back on track.

 

Loading assets

Flattr this!

Right, I’ve been working on loading assets for a while now. Preloading (caching) assets was very easy to do – the problem was that the original system for loading assets had flaws in it.

It relied on using the outdated packingslips.dat archive for figuring out where assets were located, which was causing issues with files that had complicated paths.

Therefore I’m now working on a tool, Mr. Shipper, that can generate XML files for the assets that are currently being used, as well as corresponding C# files with structures containing the IDs for these assets.

The process was going swimmingly, but then we were once again stumped by Maxis’ unique development practices. We assumed that since an unsigned integer can hold 4,294,967,295 unique values, it would be enough to map each asset in the game. And it would! The game contains a little over 35,000 assets.

That is but a few percent of the total amount of assets one would need to fill a uint’s total range. But somehow, the geniouses at Maxis decided to reuse FileIDs for different files, so that a FileID has to be combined with its TypeID (which is also an unsigned integer) to be unique. That means that to map an asset to a unique ID in the game, you need an unsigned long, which is 8 bytes rather than 4. In other words, each ID for each asset takes twice as much memory for no particular reason at all!

Monogame?

Flattr this!

Right, I haven’t made a new post in a while. Don’t fret though, development is still going strong!

Right now, we are working on fixing the loading system. Assets are now cached, which means loading should be much faster and smoother.

Development on the new phone didn’t go according to plan, so was scrapped.

Now, to the real point of this post: In Windows 8, XNA is dead. XNA apps can still run in Windows 8, which is good. This game will most likely never reach any portable devices anyway. However, the fact that the Metro system in Windows 8 doesn’t support XNA natively makes me worry about the future (looking ahead to Windows 9).

I’m going to install Windows 8 on my laptop, which rules out development for me on that. Therefore, I’m thinking of switching to MonoGame. I already tested and confirmed that it is possible to convert the codebase to XNA 4, and since MonoGame is essentially XNA 4 reproduced from the ground up using Metro-compatible code, it should be ideal for my purposes.

I’ll keep you posted!

 

Rendering on Android

Flattr this!

Because I still haven’t really made any progress on rendering 3D with XNA, and just got a new phone (HTC Wildfire S), I’ve decided to branch out technology development to Android using OpenGL ES (Embeded Systems). It is a somewhat unlikely branch (because the likelyhood of Project Dollhouse being ported to Android is slim to none), but my hope is that I’ll be able to learn something that I can apply to XNA.

So far all I have is a spinning triangle using a texture from TSO, but I haven’t really gotten started on the programming yet because I spent quite a while setting up the development environment. I’ll update this post with pictures as soon as I have something interesting to show!

Afr0 Games is now also on Twitter! Come follow us!

Tension is rising…

Flattr this!

Have you ever tried hammering a nail into a wall using nothing but a rock, with both hands tied firmly behind your back?
That’s just about how it feels trying to do any 3D operations in XNA that are even remotely advanced. I have now tried for several months to set up skeletal animation, and am now in a situation where I’ve discovered that XNA even has a built-in function for parts of what I’ve been trying to do (transforming a set of vertices based on a bone’s location in 3D space), but that this is completely useless so long as I am unable to position the skeleton’s bones correctly in 3D space.
My current theory is that the bones’ locations (which are made up of a rotation and a translation) are stored as OpenGL-native coordinates. Vitaboy (the rendering system used by The Sims and The Sims Online) was originally written in Direct3D, but it isn’t unthinkable that it was ported to OpenGL for TSO. I have tried every possible matrix and vector based operation on the bones’ translations and rotations that I could possibly think of, but nothing seems to help.
I have been putting this off for what feels as long as possible, and I’m giving myself another week, but if I haven’t made any significant progress within that timeframe I’m switching to OpenGL.
I have already tried SFML (Simple Fast Multimedia Library – an engine that allows you to use OpenGL calls and combine them with calls to the engine), but it applies some kind of funky custom world matrix that fucks up animation and rendering.
The reason I’ve been putting this off for so long is that I know that rewriting the GUI-system in OpenGL is going to take time and be a learning process all of its own, but my current consensus is that it’ll take less time than trying to fight with XNA in order to make it do something it obviously wasn’t meant to do.
I am allowing comments on this post in the hope that I can get some bright ideas and/or support.

PHP and HTTP

Flattr this!

So, after a tip from Jayenkai over at [url=http://www.socoder.com]Socoder[/url], I decided to use a webserver after all, but I wrote a couple of PHP scripts to take care of finding the manifests.

One, called “patch.php”, is the most important. It sends the right manifest(s) to a client based on what version is passed to it.


<?php

/*The contents of this file are subject to the Mozilla Public License Version 1.1
(the "License"); you may not use this file except in compliance with the
License. You may obtain a copy of the License at http://www.mozilla.org/MPL/

Software distributed under the License is distributed on an "AS IS" basis,
WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for
the specific language governing rights and limitations under the License.

The Original Code is patch.php.

The Initial Developer of the Original Code is
Mats 'Afr0' Vederhus. All Rights Reserved.

Contributor(s): ______________________________________.
*/

include 'createzip.php';

if(isset($_GET['Version']))
{
	$ClientVersion = intval(htmlspecialchars($_GET['Version']));
	$NewManifest = "";

	if($handle = opendir(realpath('./patches/')))
	{
		//Find out if there are any manifests newer than the client's version.
		while (false !== ($entry = readdir($handle)))
		{
        		if(intval(str_replace('.manifest', '', $entry)) > $ClientVersion)
			{
				$NewManifest = realpath('./patches') . '/' . $entry . '.manifest';
				break;
			}
		}

		if(empty($NewManifest) || strcmp($NewManifest, realpath('./patches')) == 0)
		{
			header('Content-Description: No New Manifest');
			header('Content-Type: text/html; charset=utf-8');
			echo('<html><body><p>No new manifest!</p></body></html>');
		}
		else
		{
			//Open the new manifest and check if it has a child, which means an
			//incremental update needs to take place...
			$FileHandle = fopen($NewManifest, 'r');

			$Parent = fgets($FileHandle);
			$CurrentChild = fgets($FileHandle);
			$Path = realpath('./patches');

			$Parent = str_replace('Parent=', '', $Parent);
			$Parent = str_replace('"', '', $Parent);
			$Parent = trim($Parent);
			$CurrentChild = str_replace('Child=', '', $CurrentChild);
			$CurrentChild = str_replace('"', '', $CurrentChild);
			$CurrentChild = trim($CurrentChild);

			fclose($FileHandle);
			
			//This is a compromise. Instead of checking for all childs, just check for the first one,
			//and let the client handle the other children (if any).
			if(empty($CurrentChild) !== true && empty($Parent) !== true)
			{
				$Parent = $Path . '/' . $Parent;
				$CurrentChild = $Path . '/' . $CurrentChild;

				$FilesToZip = array($CurrentChild, $NewManifest);
				$ZipName = './' . strval(rand()) . '.zip';
				create_zip($FilesToZip, $ZipName);
				
				//Client should check the Content-Description header to figure out what to expect.
				header('Content-Description: Zipped File Transfer');
    				header('Content-Type: application/octet-stream');
    				header('Content-Disposition: attachment; filename='.basename($ZipName));
    				header('Content-Transfer-Encoding: binary');
    				header('Expires: 0');
    				header('Cache-Control: must-revalidate, post-check=0, pre-check=0');
    				header('Pragma: public');
    				header('Content-Length: ' . filesize($ZipName));
    				ob_clean();
    				flush();
    				readfile($ZipName);

				unlink($ZipName);
				exit();
			}
			//There was a child, but no parent.
			else if(empty($CurrentChild) !== true && empty($Parent) == true)
			{
				$CurrentChild = $Path . '/' . $CurrentChild;

				$FilesToZip = array($CurrentChild, $NewManifest);
				$ZipName = './' . strval(rand()) . '.zip';
				create_zip($FilesToZip, $ZipName);
				
				//Client should check the Content-Description header to figure out what to expect.
				header('Content-Description: Zipped File Transfer');
    				header('Content-Type: application/octet-stream');
    				header('Content-Disposition: attachment; filename='.basename($ZipName));
    				header('Content-Transfer-Encoding: binary');
    				header('Expires: 0');
    				header('Cache-Control: must-revalidate, post-check=0, pre-check=0');
    				header('Pragma: public');
    				header('Content-Length: ' . filesize($ZipName));
    				ob_clean();
    				flush();
    				readfile($ZipName);

				unlink($ZipName);
				exit();
			}
			//There was a parent, but no child.
			else if(empty($Parent) !== true && empty($CurrentChild) == true)
			{
				$Parent = $Path . '/' . $Parent;

				$FilesToZip = array($Parent, $NewManifest);
				$ZipName = './' . strval(rand()) . '.zip';
				create_zip($FilesToZip, $ZipName);
				
				//Client should check the Content-Description header to figure out what to expect.
				header('Content-Description: Zipped File Transfer');
    				header('Content-Type: application/octet-stream');
    				header('Content-Disposition: attachment; filename='.basename($ZipName));
    				header('Content-Transfer-Encoding: binary');
    				header('Expires: 0');
    				header('Cache-Control: must-revalidate, post-check=0, pre-check=0');
    				header('Pragma: public');
    				header('Content-Length: ' . filesize($ZipName));
    				ob_clean();
    				flush();
    				readfile($ZipName);

				unlink($ZipName);
				exit();
			}
			//There was neither a parent nor a child. This should really never occur.
			else
			{
				header('Content-Description: File Transfer');
    				header('Content-Type: application/octet-stream');
    				header('Content-Disposition: attachment; filename='.basename($NewManifest));
    				header('Content-Transfer-Encoding: binary');
    				header('Expires: 0');
    				header('Cache-Control: must-revalidate, post-check=0, pre-check=0');
    				header('Pragma: public');
    				header('Content-Length: ' . filesize($NewManifest));
    				ob_clean();
    				flush();
    				readfile($NewManifest);

				exit();
			}
		}
	}
}
else
{
	header('Content-Description: Invalid Request');
	header('Content-Type: text/html; charset=utf-8');
	echo('<html><body><p>Invalid request!</p></body></html>');
}
?>

This code depends on another script I found while rummaging through Google:

<?php
//From: http://davidwalsh.name/create-zip-php
function create_zip($files = array(),$destination = '',$overwrite = false) {
  //if the zip file already exists and overwrite is false, return false
  if(file_exists($destination) && !$overwrite) { return false; }
  //vars
  $valid_files = array();
  //if files were passed in...
  if(is_array($files)) {
    //cycle through each file
    foreach($files as $file) {
      //make sure the file exists
      if(file_exists($file)) {
        $valid_files[] = $file;
      }
    }
  }
  //if we have good files...
  if(count($valid_files)) {
    //create the archive
    $zip = new ZipArchive();
    if($zip->open($destination,$overwrite ? ZIPARCHIVE::OVERWRITE : ZIPARCHIVE::CREATE) !== true) {
      return false;
    }
    //add the files
    foreach($valid_files as $file) {
      $zip->addFile($file,$file);
    }
    //debug
    //echo 'The zip archive contains ',$zip->numFiles,' files with a status of ',$zip->status;
    
    //close the zip -- done!
    $zip->close();
    
    //check to make sure the file exists
    return file_exists($destination);
  }
  else
  {
    return false;
  }
}
?>

The ‘patch.php’ script can be utilized by clients as such;

http://www.webserver.com/patch.php?Version=ClientVersion

Lastly there’s the ‘getmanifest.php’ script which simply retrieves a specific manifest;

<?php
/*The contents of this file are subject to the Mozilla Public License Version 1.1
(the "License"); you may not use this file except in compliance with the
License. You may obtain a copy of the License at http://www.mozilla.org/MPL/

Software distributed under the License is distributed on an "AS IS" basis,
WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for
the specific language governing rights and limitations under the License.

The Original Code is getmanifest.php.

The Initial Developer of the Original Code is
Mats 'Afr0' Vederhus. All Rights Reserved.

Contributor(s): ______________________________________.
*/

if(isset($_GET['Manifest']))
{
	$RequestedManifest = htmlspecialchars($_GET['Manifest']);
	$FoundManifest = "";

	if($handle = opendir('./patches/'))
	{
		//Find the requested manifest...
		while (false !== ($entry = readdir($handle)))
		{
        		if($entry === $RequestedManifest)
			{
				$FoundManifest = './patches/' . $entry;
				break;
			}
		}

		if(empty($FoundManifest) || strcmp($FoundManifest, './patches') == 0)
		{
			header('Content-Description: Invalid Request');
			header('Content-Type: text/html; charset=utf-8');
			echo('<html><body><p>No manifest by that name exists!</p></body></html>');
		}
		else
		{
			header('Content-Description: File Transfer');
    			header('Content-Type: application/octet-stream');
    			header('Content-Disposition: attachment; filename='.basename($FoundManifest));
    			header('Content-Transfer-Encoding: binary');
    			header('Expires: 0');
    			header('Cache-Control: must-revalidate, post-check=0, pre-check=0');
    			header('Pragma: public');
    			header('Content-Length: ' . filesize($FoundManifest));
    			ob_clean();
    			flush();
    			readfile($FoundManifest);

			exit();
		}
	}
}
else
{
	header('Content-Description: Invalid Request');
	header('Content-Type: text/html; charset=utf-8');
	echo('<html><body><p>Invalid request!</p></body></html>');
}
?>

It can be utilized as such;

http://www.webserver.com/getmanifest.php?Manifest=ManifestVersion.manifest

I also created a tool for generating these manifests. The important thing, however, is that the two first lines of a manifest are stored as plain ASCII strings so they can be read by ‘patch.php’. Aside from that, manifests can look however you want them to.
Here’s a sample manifest generated by Manifestation;

Child=””
NumFiles=48
&DLLs\Aries.dll MD5: ?5??????
I?[/?
*DLLs\authlogin.dll MD5: ?????
(HitListsTemp.dat MD5: ~l?{??JL,?2?9?
.DLLs\HitMp3DecodeD.dll MD5: ????psCX?p’??2%6
&DLLs\ijl10.dll MD5: ?r?q.GcP??
1DLLs\InternetServiceD.dll MD5: ? y?fU?|[?ro???
8DLLs\TSONetServiceSimClientD.dll MD5: !???m??>`

The added MD5s after filenames are checksums that can be used by clients to verify file integrity.
If you’re looking for a general-purpose way to patch your game(s), I hope I’ve given you some food for thought!

FTP vs. HTTP

Flattr this!

Right. I just tried getting the FTP server I’ve been working on to work with FileZilla for the umpteenth time. It still doesn’t work, no matter what I do.

Unfortunately, the FTP-protocol is extremely badly documented (aside from the documents available from IETF, which are extremely terse, contains unneeded bloat and are generally just downright bad). I’ve been able to make my way up to and including the LIST and/or MLSD command (the MLSD being a newer extension command), but when trying to transfer the data containing the directory listing for the root directory, FileZilla never acknowledges that the data was received! Instead it times out and decides that there was a problem receiving the directory listing.

So, I’ve been thinking, would it make sense to just switch to HTTP for transferring patches to clients? It is starting to look extremely tempting. Arguably, I could code my way around the FTP-protocol by introducing non-standard protocol “extensions”, but that would defeat the purpose of using FTP in the first place (namely that it’s a pre-designed filetransfer protocol).

I don’t know if I’ll have the time tomorrow (I should study), but one of these days I’m going to install a copy of Apache on my box and see about making my patch client communicate with it.

So far I’ve been thinking of two possible solutions for a directory structure on the server:

  • Dumping a manifest  and all files into the root folder. This would have the downside of making it near-impossible to do incremental updates, unless the manifest contained information about which files belonged to which update.
  • Have a manifest in the root folder that points to different versions contained in separate directories. This seems like the best way to go, as it would enable incremental updates without making the manifest needlessly complex.

Once I’ve been able to write some basic HTTP code for the patch client, I’ll make another post with pictures of the client to inform of the progress.