sizeof vs Marshal.SizeOf !!!

There are two facilities in C# to determine the size of a type - sizeof operator and Marshal.SizeOf method. Let me discuss what they offer and how they differ. Pardon me if I happen to ramble a bit.

Before we settle the difference between sizeof and Marshal.SizeOf, let us discuss why would we want to compute the size of a variable or type. Other than academic, one typical reason to know the size of a type (in a production code) would be allocate memory for an array of items; typically done while using malloc. Unlike in C++ (or unmanaged world), computing the size of a type definitely has no such use in C# (managed world). Within the managed application, size does not matter; since there are types provided by the CLR for creating\managing fixed size and variable size (typed) arrays. And as per MSDN, the size cannot be computed accurately. Does that mean we don't need to compute the size of a type at all when working in the CLR world? Obviously no, else I would not be writing this post.

Value Types: User-defined value (and reference types) are composed of the primitive value types exposed by the compiler, most of which exist as keywords – int, bool, char, long, double etc. Since the primitive value types are exposed by the compiler, their sizes are (pre-)defined by the compiler (based on the platform on which the CLR runs). The compiler allows querying the sizes of the primitive value types the sizeof operator. The sizeof operator returns the size of the type in bytes as allocated by the CLR (on the current platform). Refer MSDN for the sizes of primitive types.

However, the sizeof cannot be freely used with user-defined value types (struct) but only if the following conditions are true:-

  • The size of the struct is requested from within an unsafe block.

  • The struct does not contain a reference type as its member.
    Since the size of a reference type cannot be computed (see Reference Types below), the size of the struct cannot be computed too.

  • The struct is not a generic value type.
    sizeof should be imagined as a compile-time construct. That implies the type for which the size is queried should be known at compile time. When we say sizeof(GenValueType), the (closed) type definition GenValueType is not available at compile time but only at runtime. Hence the compiler does not allow computing the size of a generic value type. But the size of an instance of the same closed type may be determined. But ideally it does not sound convincing to me because the compiler uses the MSIL sizeof instruction to computer the size, instead of hard coding the size (as is done for primitive types).

Besides, the subtle and bitter thing is that the size depends on other factors such as the pack size used (StructLayout.Pack) or character set (StructLayout.CharSet) applied on the type definition or the fixed size specified (StructLayout.Size). Unlike in C++, sizeof accepts only a (closed) type known at compile time and not variables.

Reference Types: The sizeof operator cannot be used on reference types. As per MSDN, the size can be either misleading or meaningless for reference types. Consider a class (reference type) SomeClass containing a char and a string. Reference types are basically (C++) pointer like. When computing the size of SomeClass, which aspect of the string member should be consider - the reference (4 bytes) or the value (n bytes)? Also, every .NET object incurs a 16 (I guess) byte header overhead. Should we consider the header size too and the same question applies here too - 16 bytes or the mazy data structure that the header actually refers to. So for such reasons, it does not make sense to determine the size of a reference type using sizeof (at least at compile time). The other way of putting this is sizeof works only for POD types.

Given all this uncertainty in computing the size of a type (using sizeof), will there ever be a need then? In a broader sense, there is one situation. That is when the data is passed out of the managed application - Interop or custom serialization and such. For example, the managed application might want to allocate unmanaged memory for creating\filling a data structure for calling a native API, which takes the data structure as its input or would be populating it with output.

Let us enter the second half (or the better half) - Marshal.SizeOf. Unlike sizeof (C# keyword), this one is offered from the BCL. This method returns the size (in bytes) of the type or its instance if it had to exist in the unmanaged world. This method has two overloads - one taking the type as input and the other an instance. Let us say we want to allocate some memory in the unmanaged heap to call a native API (SendMessage or VirtualAlloc or ReadProcessMemory). In many cases, the amount of memory to be allocated is the equivalent of a Win32 structure -LVITEM, STARTUPINFO or one such. In such situations, Marshal.SizeOf method has to be used { int x = Marshal.SizeOf(typeof(LVITEM)); }, which returns the size of the structure depending on the StructLayoutAttribute applied.

Can Marshal.SizeOf method be used on reference and value types? It can be used on any value type but will throw an exception at runtime if the value type contains a reference type. And the error makes sense - Type cannot be marshaled as an unmanaged structure; no meaningful size or offset can be computed. Otherwise, it can be used with primitive or user-defined value types. It is allowed to be used with reference types only if the type layout is specified to be LayoutKind.Sequential or LayoutKind.Explicit; else the same exception above will be thrown at runtime.

It is possible that the size returned by sizeof and Marshal.SizeOf are different, as with the case of char. sizeof(char) is 2 since CLR is an Unicode beast. Marshal.SizeOf(char) will return 1 since a char in the unmanaged world takes up one byte. However, Marshal.SizeOf(SomeStruct) may report to that its char member consumes two bytes (by default) or made to take up one byte (if the StructLayout.CharSet=CharSet.Ansi).

Do I need to conclude something?

Curious Case Of Anonymous Delegates !!!

Senthil has left us thrilled in his new post, and also inspired me to write about the topic. Although, anonymous delegates have become a mundane stuff amongst programmers, there is still these subtle stuff left unexplored. Alright, let us try to answer Senthil's question before he unravels the mystery in his next post.

A delegate is identified by its target. The target is the method to be executed on a delegate invocation and its associated instance or type. If the target is an instance method, the delegate preserves the target method pointer and the object instance. If the target is a static method, the delegate preserves the target method pointer and the type to which it belongs. So when a code like the one below to register a method to an event (or multicast delegate) is executed, a delegate object (EventHandler here) with the target information embedded is created and added to the invocation list of the event (or multicast delegate, KeyPressed here).

class SomeForm
{
   private Control control = new Control();

   public void OnFormLoad(object sender, EventArgs args)
   {
      control.KeyPressed += new EventHandler(OnKeyPressed);
   }

   // Rest of the code omitted to be succinct
};

Likewise, when unregistering the method handler, a new (EventHandler) delegate object is created with the same target information as above. As said earlier, a delegate is identified by its target. In other words, the Equals override on the delegate uses the target information for comparing two delegate objects. Hence in the following code that unregisters the method handler, the invocation list is searched for a delegate instance with the specified target information (Method: OnKeyPressed, Instance: SomeForm instance).

In the case of anonymous delegates, the compiler transforms the inline method code into a

  • static method, if the inline method code does not use any of the class's instance members or local variables or if it uses only the static members of the class.
  • instance method, if the inline method code uses at least one class member, any or no static members, and no local variables.
  • class with a method that represents the inline method code, if the inline method code uses local variables no matter whether it uses the class members or not.

Those might not be the extensive set of rules but sure are enough for our discussion. Given the following questionable code,

public EventHandler IfEnabledThenDo(EventHandler actualAction)
{
   return (sender, args) => {
      if (args.Control.Enabled)
      {
         actualAction(sender, args);
      }
   };
}

public void Initialize()
{
   control.KeyPressed += IfEnabledThenDo(control_KeyPressed);
}

public void Destroy()
{
   control.KeyPressed -= IfEnabledThenDo(control_KeyPressed);
}

we realize, without doubt, that the anonymous delegate (returned by IfEnabledThenDo) would be transported into a compiler generated anonymous class. Later when IfEnabledThenDo is called for registering\unregistering the method handler, an instance of anonymous class is created and the (EventHandler) delegate is returned. And here lies the subtlety. Although the delegate from IfEnabledThenDo targets the method inside the anonymous class, the instance preserved as a part of the target information are different during registration and un-registration. In other words, the target method of the delegate returned by IfEnabledThenDo belong to different instances of the anonymous class. Hence the pretty code to unregister the (key pressed) method handler would not be actually unregistering since there would be a delegate previously registered in the invocation list of the (KeyPressed) event with the target instance same as the one used in the unregistration line of code. Very subtle!

Usually the hand written code tends to keep the registration and unregistration of the method handlers in the same class and so belong to the respective instances. Not so when you like watching the compiler magic.

Let us wait and see what Senthil says.

finally and Return Values !!!

Let us read some code:-

int SomeMethod()
{
    int num = 1;

    try
    {
        num = 5;
        return num;
    }
    finally
    {
        num += 5;
    }
}

What is the return value of SomeMethod? Some anonymous guy asked that question in the code project forum, and it has been answered. I am writing about it here because it is interesting and subtle. One should not be surprised when people misinterpret finally. So let us take a guess, 10 (i = 5, then incremented by 5 in the finally block).

It is not the right answer; rather SomeMethod returns 5. Agreed that finally is called in all cases of returning from SomeMethod but the return value is calculated when it is time to return from SomeMethod, normally or abnormally. The subtlety lies not in the way finally is executed but in the return value is calculated. So the return value (5) is decided when a return is encountered in the try block. The finally is just called for cleanup; and the num modified there is local to SomeMethod. So make the return value 10, it is no use being hasty making SomeMethod return from the finally block. Because returning from finally is not allowed. (We will talk about it later why returning from catch block is a bad practice and why can't we return from finally block).

Had such modifications been done on a reference type, they would have been visible outside of SomeMethod, although the return value may be different. For instance,

class Num
{
    public int _num = 0;
};

int SomeMethod()
{
    Num num = new Num();

    try
    {
        num._num = 5;
        return num._num;
    }
    finally
    {
        num._num += 5;
    }
}

So in the above case, the return value is still 5, but the Num._num would have been incremented to 10 when SomeMethod returns. So reflecting shows that our code is transformed as follows by the compiler, where the CS$1$0000 is our return value.

private static int SomeMethod(Num num)
{
    int CS$1$0000;

    try
    {
        num._num = 5;
        CS$1$0000 = num._num;
    }
    finally
    {
        num._num += 5;
    }

    return CS$1$0000;
}

Given that we have clarified ourselves about finally, we should be writing the code as transformed by the compiler because returning from try and catch blocks is not a good practice.

Type Safe Logger

Sanjeev and I have published an article - Type Safe Logger For C++ - at CodeProject. Every bit of work is tiresome or little ugly in C++. So is logging - writing application diagnostics to console, file etc. The printf style of outputting diagnostics is primitive and not type safe. The std::cout is type safe but does not have a format specification. Besides that, printf and std::cout know to write only to the console. So we need a logging mechanism that provides a format specification, is type safe and log destination transparent. So we came up with this new Logger to make C++ programmers happy.

Following is a short introduction excerpt of the article:-

Every application logs a whole bunch of diagnostic messages, primarily for (production) debugging, to the console or the standard error device or to files. There are so many other destinations where the logs can be written to. Irrespective of the destination that each application must be able to configure, the diagnostic log message and the way to generate the message is of our interest now. So we are in need of a Logger class that can behave transparent to the logging destination. That should not be a problem, it would be fun to design that.......Read more.
As always your comments are most valuable.

And oh, Happy Logging!

Simple Array Class For C++

This is a simple array like class for C++, which can be used as a safe wrapper for accessing a block of memory pointed by a bare pointer.

#pragma once

template<typename T> class Array
{
private: T* _tPtr;
private: size_t _length;
private: bool _isOwner;

public: Array(size_t length, bool isOwner = true)
           : _isOwner(isOwner)
        {
           _length = length;
           _tPtr = new T[length]; 
        }

public: Array(T* tPtr, size_t numItems, bool isOwner = true)
           : _isOwner(isOwner)
        {
           if (NULL == tPtr)
           {
              throw std::exception("Specified T* pointer is NULL.");
           }

           this->_length = numItems;
           this->_tPtr = tPtr;
        }

public: template<typename TSTLContainerType>
        Array(const TSTLContainerType& stlContainer, bool isOwner) : _isOwner(isOwner)
        {
           _length = stlContainer.size();
           _tPtr = new T[_length];

           int index = 0;
           for (TSTLContainerType::const_iterator iter = stlContainer.begin();
              iter != stlContainer.end();
              ++iter, ++index)
           {
              _tPtr[index] = *iter;
           }
        }

public: ~Array()
        {
           if (IsOwner())
           {
              delete _tPtr;
           }
        }

public: T& operator[](size_t index) const
        {
           return GetItem(index);
        }

public: T& operator[](int index) const
        {
           return GetItem(static_cast<size_t>(index));
        }

        // Gets a copy of the value at the specified index
public: T GetValue(size_t index)
        {
           return GetItem(index);
        }

public: operator T* const() const
        {
           return _tPtr;
        }

public: T* const Get() const
        {
           return _tPtr;
        }

public: operator const T* const()
        {
           return _tPtr;
        }

public: int Length() const
        {
           return static_cast<int>(this->_length);
        }

public: int Size() const
        {
           return _length * typeof(T);
        }

public: bool IsOwner() const
        {
           return this->_isOwner;
        }

public: void CopyTo(T* tPtr, size_t copySize)
        {
           memcpy(tPtr, this->_tPtr, copySize);
        }

        // Gets a reference to the item (T) at the specified index
private: T& GetItem(size_t index) const
         {
            const size_t length = static_cast<size_t>(Length());
            if  (index > length)
            {
               throw std::exception("Index out of bounds");
            }

            return _tPtr[index];
         }
};

This class is also available at KenBase.

Disclaimer: It is not aimed to replace the STL containers.

Passing CComPtr By Value !!!

This is about a killer bug identified by our chief software engineer in our software. What was devised for ease of use and write smart code ended up in this killer defect due to improper perception. Ok, let us go!

CComPtr is a template class in ATL designed to wrap the discrete functionality of COM object management - AddRef and Release. Technically it is a smart pointer for a COM object.

void SomeMethod()
{
   CComPtr siPtr;
   HRESULT hr = siPtr.CoCreateInstance(CLSID_SomeComponent);
   siPtr->MethodOne(20, L"Hello");
}

Without CComPtr, the code wouldn't be as elegant as above. The code would be spilled with AddRef and Release. Besides, writing code to Release after use under any circumstance is either hard or ugly. CComPtr automatically takes care of releasing in its destructor just like std::auto_ptr. As a C++ programmer, we must be able to appreciate the inevitability of the destructor and its immense use in writing smart code. However there is a difference between pointers to normal C++ objects and pointers to COM objects; CComPtr and std::auto_ptr. When you assign one auto_ptr to another, the source is no more the owner of the object pointing to. The ownership is transferred to the destination. Whereas when a CComPtr is assigned to another, the reference count of the target COM object increases by one. And the two CComPtrs point to the same COM object. Changes made via one CComPtr object can be realized when the object is accessed via the other CComPtr. Release must be called on each CComPtr instance (to completely release the COM object). All fine, lets us see some code.

void SomeOtherMethod()
{
   CComPtr aPtr;
   InitAndPopulateObject(aPtr);

   int itemCount = 0;
   HRESULT hr = aPtr->GetCount(&itemCount);
   _ASSERTE(SUCCEEDED(hr));

   for (int i = 0; i < itemCount; ++i)
   {
      TCHAR szBuffer[128] = { 0 };
      sprintf_s(szBuffer, sizeof(szBuffer), "Key%ld", i);
      CComBSTR bstrKey(szBuffer);

      int iValue = 0;
      hr = aPtr->GetItem(bstrKey, &iValue);
      _ASSERTE(SUCCEEDED(hr));

      std::cout << bstrKey << " - " << iValue;
   }
}

void InitAndPopulateObject(CComPtr bPtr)
{
   HRESULT hr = bPtr.CoCreateInstance(CLSID_Hashtable);
   
   _ASSERTE(SUCCEEDED(hr));

   for (int i = 0; i < 100; ++i)
   {
      TCHAR szBuffer[128] = { 0 };
      sprintf_s(szBuffer, sizeof(szBuffer), "Key%ld", i);
      bPtr->Add(szBuffer, i);
   }
}

CComPtr saved a whole of code as explained above. But my application was always crashing in SomeOtherMethod when GetCount method is called on the COM object initialized one line above. So I am passing a CComPtr to InitAndPopulateObject, which is supposed to create me my COM object and fill it with some information I expect. Since I am passing a CComPtr, a return value is not needed. Looks fine, but the application crashed.

People are often misled with many things in programming mostly because they stick to the prime way of its use. CComPtr, in most cases, is used for creating a COM object, passed around across various sections in the code where AddRef and Release is done under the covers until the COM object dies a pleasant death. People tend to forget that the member in CComPtr (named poorly as p) is the one that is actually pointing to the COM object. So aPtr.p, whose value is 0x0000 (NULL), is passed by value and copied to bPtr.p. When the COM object is created using bPtr, it is bPtr.p ,which is assigned the COM object's address, say 0x23456789; whereas aPtr.p remains NULL even after InitAndPopulateObject returns. Hence the application was crashing because of null pointer access.

The problem might be obvious in the above few lines of clear code. It sure was very tough to locate and reason it in our huge code base.

OrderedThreadPool - Task Execution In Queued Order !!!

I would not want to write chunks of code to spawns threads and perform many of my background tasks such as firing events, UI update etc. Instead I would use the System.Threading.ThreadPool class which serves this purpose. And a programmer who knows to use this class for such cases would also be aware that the tasks queued to the thread pool are NOT dispatched in the order they are queued. They get dispatched for execution in a haphazard fashion.

In some situations, it is required that the tasks queued to the thread pool are dispatched (and executed) in the order they were queued. For instance, in my (and most?) applications, a series of events are fired to notify the clients with what is happening inside the (server) application. Although the events may be fired from any thread (asynchronous), I would want them or rather the client would be expecting that the events are received in a certain order, which aligns with the sequence of steps carried out inside the server application for the requested service. So sequential execution of the queued tasks is not something one must not wish for.

Enough talking.......eat code.

using System;
using System.Collections.Generic;
using System.Diagnostics;

namespace System.Threading
{
struct ThreadPoolTaskInfo
{
public readonly WaitCallback CallbackDelegate;
public readonly object State;

public ThreadPoolTaskInfo(WaitCallback wc, object state)
{
Debug.Assert(wc != null);
CallbackDelegate = wc;
State = state;
}
}

class OrderedThreadPool
{
private Queue workItemQ = new Queue();

public void QueueUserWorkItem(WaitCallback wcbDelegate, object state)
{
lock (workItemQ)
{
workItemQ.Enqueue(new ThreadPoolTaskInfo(wcbDelegate, state));

if (workItemQ.Count == 1)
{
ThreadPool.QueueUserWorkItem(LoopWork);
}
}
}

private void LoopWork(object notUsed)
{
WaitCallback wcb = null;
object state = null;

lock (workItemQ)
{
if (workItemQ.Count == 0)
{
return;
}

ThreadPoolTaskInfo tptInfo = workItemQ.Dequeue();
state = tptInfo.State;
wcb = tptInfo.CallbackDelegate;
Debug.Assert(wcb != null);
}

try
{
wcb(state);
}
finally
{
ThreadPool.QueueUserWorkItem(LoopWork, notUsed);
}
}
}
}

The above class wraps the System.Threading.ThreadPool and offers the facility of execution of tasks in the order they are queued. Hope that is useful!

Settling Casting Restrictions !!!

Remember the Casting Restrictions we discussed a while back, let us settle that now. So we have some code like this:

int i = 100;
object obj = i;
long l = (long)obj;

And an invalid cast exception while casting 'obj' to long. It is obvious that we are not changing the value held by obj, but just reading it. Then why restrict such casting. Let us disassemble and see what we got.

    .locals init (
        [0] int32 i,
        [1] object obj,
        [2] int64 l)
    L_0000: nop
    L_0001: ldc.i4.s 100
    L_0003: stloc.0
    L_0004: ldloc.0
    L_0005: box int32
    L_000a: stloc.1
    L_000b: ldloc.1
    L_000c: unbox.any int64
    L_0011: stloc.2
    L_0012: ret

Oh, there we see something interesting - unbox. So the C# compiler uses the unbox instruction to retrieve the value from obj while casting; it does not use Convert.ToInt64 or similar mechanism. That is why the exception was thrown.

From MSDN:

Unboxing is an explicit conversion from the type object to a value type or from an interface type to a value type that implements the interface. An unboxing operation consists of:
  • Checking the object instance to make sure it is a boxed value of the given value type
  • Copying the value from the instance into the value-type variable

So we are blown at step 1 of the unbox operation. Let us play with what we have for now, and stop bugging why was unbox meant to be like that.

The WD Anti-Propaganda Campaign !!!

Thanks to the internet. If nobody else bothers or understands what loss of data means, you can shout it aloud here. I lost 500GB of data - every moment of my personal and professional life captured in bits and bytes.

It is a Western Digital Premium Edition external hard disk (USB/Firewire). I bought it despite my friend warning of bad sectors and hardware issues that WD is known to have. As with any story, one fine morning, I was copying some songs, pictures from my pen drive to the hard disk. All of a sudden, the hard disk and my laptop hung up. I restarted the system thinking I would make fresh start. But to my dismay, all my drives on the hard disk had vanished like dust. I tried connecting and reconnecting a few times, the drives showed up once or twice like a sick man's last few breadths.

The other similar incident was the hard disk crash at my office last month. It was also a WD 160GB hard disk (IDE). And it took with it more than 5 years of email storage, project documents, code and other official data. My friend's warning until it happened seemed to be a rumour but is proven now. I think it is fair on my part to make polite anti-propoganda against WD.

The drive not only took with it gobs and gobs of data, it took the proof of my existence.