Groups | Blog | Home
all groups > dotnet performance > september 2004 >

dotnet performance : how to: decrease the size of serialized objects?



Bob Grommes
9/24/2004 4:11:44 PM
There's nowhere near enough information or context in your question to
answer it properly, but in general, use a binary formatter as opposed to
serializing as XML -- assuming your situation allows you to do so (for
example, both the serializer and deserializer are .NET applications).

--Bob

[quoted text, click to view]

assaf
9/25/2004 12:50:26 AM
hi all

when i serialize objects,
they seem to get 100 times the original size.

how can i decrease the size of serialized objects?


assaf


Piotrek
9/25/2004 11:22:22 AM
Hi

What kind of objects are you serialising?

Justin Rogers
10/1/2004 5:33:56 PM
The BinaryFormatter does a lot of things in a verbose manner. For one, it stores
a bunch of strings identifying the types and fields for reconstruction later. It
also
does very little compression of numeric data (I say very little because they do
use
7 bit encoded integers for string lengths).

If you want to decrease the size you can run a custom compression routine after
you've done the formatting. If you plan on doing this you might as well write
your
own serialization routines. .NET serialization was meant to solve the problem of
remoting and it is extremely poor at network data management for basic data
sharing in programs or for network gaming. I'm assuming an application here on
your part, and I could be completely wrong, but perhaps you aren't looking for
serialization.

--
Justin Rogers
DigiTec Web Consultants, LLC.
Blog: http://weblogs.asp.net/justin_rogers

[quoted text, click to view]
assaf
10/2/2004 1:10:06 AM
hi bob

the following code can be run.
u will see that an empty packet is 131 bytes.
packets filled with minimal stuff come out to almost a kilobyte each.

assaf

//// start of code

using System.IO;
using System.Runtime.Serialization;
using System.Runtime.Serialization.Formatters.Binary;
using System;
using System.Drawing;
using System.Collections;
using System.ComponentModel;
using System.Windows.Forms;
using System.Data;

namespace Serialize
{
/// <summary>
/// Summary description for Form1.
/// </summary>
public class Form1 : System.Windows.Forms.Form
{
private System.Windows.Forms.Button button1;
/// <summary>
/// Required designer variable.
/// </summary>
private System.ComponentModel.Container components = null;

public Form1()
{
//
// Required for Windows Form Designer support
//
InitializeComponent();

//
// TODO: Add any constructor code after InitializeComponent call
//
}

/// <summary>
/// Clean up any resources being used.
/// </summary>
protected override void Dispose( bool disposing )
{
if( disposing )
{
if (components != null)
{
components.Dispose();
}
}
base.Dispose( disposing );
}

#region Windows Form Designer generated code
/// <summary>
/// Required method for Designer support - do not modify
/// the contents of this method with the code editor.
/// </summary>
private void InitializeComponent()
{
this.button1 = new System.Windows.Forms.Button();
this.SuspendLayout();
//
// button1
//
this.button1.Location = new System.Drawing.Point(8, 8);
this.button1.Name = "button1";
this.button1.TabIndex = 0;
this.button1.Text = "button1";
this.button1.Click += new System.EventHandler(this.button1_Click);
//
// Form1
//
this.AutoScaleBaseSize = new System.Drawing.Size(5, 13);
this.ClientSize = new System.Drawing.Size(292, 273);
this.Controls.Add(this.button1);
this.Name = "Form1";
this.Text = "Form1";
this.ResumeLayout(false);

}
#endregion

/// <summary>
/// The main entry point for the application.
/// </summary>
[STAThread]
static void Main()
{
Application.Run(new Form1());
}

public interface IPacket
{
int Id{get;}
}

public interface IStatusInfo
{
int GlobalServerId {get;}
int GlobalId {get;}
int LocalId {get;}
int RoomId {get;}
string UserName {get;}
int SentenceId {get;}
string RoomName {get;}
}

[Serializable]
public class Packet : IPacket
{
#region Public Variables

#endregion Public Variables

#region Private Variables

private static int _IdCounter;

#endregion Private Variables

public readonly int Id;
public readonly int GlobalUserId;
public readonly int LocalUserId;
public readonly int ServerId;
public readonly DateTime TimeStamp;

#region Ctor
public Packet(IStatusInfo isi)
{
this.Id = Packet._IdCounter++;
this.GlobalUserId = 0; // isi.GlobalId;
this.LocalUserId = 0; // isi.LocalId;
this.ServerId = 0; // isi.GlobalServerId;
this.TimeStamp = DateTime.Now;
}

#endregion Ctor

#region IPacket Members

int IPacket.Id
{
get
{
return this.Id;
}
}
#endregion
}

[Serializable]
public class Data : Packet
{
#region Public Variables

public readonly int RoomId;
public readonly int SentenceId;
public readonly int StreamId;
public readonly int Size;

#endregion Public Variables

public Data(int size, int streamId, IStatusInfo sInfo)
: base (sInfo)
{
this.Size = size;
this.RoomId = 0; // sInfo.RoomId;
this.SentenceId = 0; // sInfo.SentenceId;
this.StreamId = streamId;
}
}

[Serializable]
public class TextPacket : Data
{
#region Public Variables

public readonly string _Text;

#endregion Public Variables

public TextPacket(string txt, int size, int streamId, IStatusInfo sInfo)
: base(size, streamId, sInfo)
{
this._Text = txt;
}
}

[Serializable]
public class SuperPacket : Packet
{
#region Private Vatiables

private readonly Hashtable _Hashtable;

#endregion Private Vatiables

#region Public Vatiables

public readonly int RoomId;

#endregion Public Vatiables

#region Ctor

public SuperPacket(IStatusInfo sInfo) : base (sInfo)
{
this._Hashtable = Hashtable.Synchronized(new Hashtable());
this.RoomId = 0; // sInfo.RoomId;
}

#endregion Ctor

public Data this[int key]
{
get
{
return (Data)this._Hashtable[key];
}
}

public void Add(Data d)
{
this._Hashtable.Add(this._Hashtable.Count, d);
}

public int Count()
{
return this._Hashtable.Count;
}
}

[Serializable]
public class EncryptPacket : Packet
{
#region Public Vatiables

public readonly int RoomId;
public readonly bool IsEncrypted;
public readonly SuperPacket SPacket;

#endregion

#region Ctor

public EncryptPacket(IStatusInfo sInfo, SuperPacket sp) : base (sInfo)
{
this.SPacket = sp;
this.RoomId = 0; // sInfo.RoomId;
}

#endregion Ctor
}

[Serializable]
private class Packet2 : Packet
{
public int i;
internal Packet2(int i)
: base(null)
{
this.i = i;
}
}

[Serializable]
private class Packet3 : Packet2
{
public int j;
internal Packet3(int i, int j)
: base(i)
{
this.i = i;
this.j = j;
}
}

private void Serialize1()
{
BinaryFormatter bf = new BinaryFormatter();
MemoryStream ms = new MemoryStream();
Packet p = new Packet(null);
bf.Serialize(ms, p);
byte[] buffer = ms.ToArray();
int bufferSize = buffer.Length;
Console.WriteLine("bufferSize: {0}", bufferSize);
foreach(byte b in buffer)
{
Console.Write((char)b);
}
}

private void Serialize2()
{
BinaryFormatter bf = new BinaryFormatter();
MemoryStream ms = new MemoryStream();
Packet2 p = new Packet2(2);
bf.Serialize(ms, p);
byte[] buffer = ms.ToArray();
int bufferSize = buffer.Length;
Console.WriteLine("bufferSize: {0}", bufferSize);
foreach(byte b in buffer)
{
Console.Write((char)b);
}
}

private void Serialize3()
{
BinaryFormatter bf = new BinaryFormatter();
MemoryStream ms = new MemoryStream();
Packet3 p = new Packet3(2, 5);
bf.Serialize(ms, p);
byte[] buffer = ms.ToArray();
int bufferSize = buffer.Length;
Console.WriteLine("bufferSize: {0}", bufferSize);
foreach(byte b in buffer)
{
Console.Write((char)b);
}
}

private void SerializeText()
{
Frank Hileman
10/2/2004 8:09:06 AM
I think he means to create your own protocol. That is, a numeric identifier
for the beginning of each packet of object data, then a known size and
format for each object and object field you send. Not only is it faster but
you no longer need for the client and server to share a common assembly --
they only share the format of the binary stream. Your packets are simple and
would not be difficult. Then you can use basic TCP/IP for communication,
which is fairly fast in .NET.

Regards,
Frank Hileman

check out VG.net: www.vgdotnet.com
Animated vector graphics system
Integrated Visual Studio .NET graphics editor


----- Original Message -----
From: "assaf" <assafwo@hotmail.com>
Newsgroups: microsoft.public.dotnet.framework.performance
Sent: Saturday, October 02, 2004 2:03 AM
Subject: Re: how to: decrease the size of serialized objects?


[quoted text, click to view]

assaf
10/2/2004 11:03:45 AM
hi justin.

when u say ' your own serialization' what do u mean?
u mean turning our data stuctures (packets)
into a byte array on our own?

or something else?


assaf


[quoted text, click to view]
assaf
10/2/2004 5:15:03 PM
hi frank.

u just gave me a very good explanation.
thank u so much.
i'll get on it right away.


assaf


[quoted text, click to view]

AddThis Social Bookmark Button