Showing posts with label Tests. Show all posts
Showing posts with label Tests. Show all posts

Wednesday, December 22, 2010

Enabling Code Coverage in VS2010 without MSTest

Just so I don’t forget how to do it…

The command-line sequence is as follows:

  1. Instrument the assemblies that you want to provide code coverage information using VSInstr.exe (32-bit version at %VSInstallDir%\Team Tools\Performance Tools\VSInstr.exe, or 64-bit version at %VSInstallDir%\Team Tools\Performance Tools\x64\VSInstr.exe) with the /COVERAGE switch.
    This will write two new files for the instrumented assembly (if PDB’s are available) named after the instrumented assembly and its PDB file with the pdb.instr extension.  The original assembly is renamed to AssemblyName.Extension.orig and the original PDB is left unchanged.

    A side effect of this instrumentation is that your assembly will now have a dependency on VSCoverX.dll (where X is 80, 90, or 100, depending on the Visual Studio version).  This dependency is what sends your coverage information to the performance profiler.
  2. Start the performance profiler using VSPerfCmd.exe (32-bit version at %VSInstallDir%\Team Tools\Performance Tools\VSPerfCmd.exe, or 64-bit version at %VSInstallDir%\Team Tools\Performance Tools\x64\VSPerfCmd.exe) so that it can begin collecting code coverage information.  The command-line is /START:COVERAGE /WAITSTART /USER:”%USERDOMAIN%\%USERNAME%” /TARGETCLR:vn.n.nnnn /OUTPUT:”pathToCodeCoverageResultFile.
  3. Perform your tests (using the same bit-ness that you’re profiling, 32-bit or 64-bit), either manually or using whatever automated testing framework you wish.
  4. Stop the performance profiler using VSPerfCmd.exe with the /SHUTDOWN switch.
  5. As a last step, rename your *.orig assemblies back to their original names, either deleting or renaming the instrumented versions.

You should now be able to open the code coverage file at pathToCodeCoverageResult in Visual Studio to examine the code coverage information, or create an XML report using VSPerfReport.exe.

Technorati Tags: ,,

Wednesday, December 01, 2010

Delegate Identities

Technorati Tags: ,

Verified some interesting behavior today that I had suspected but wasn’t entirely sure of.  It appears that a System.Delegate’s identity is determined by the object instance (or type) and method that it points to, and that the Combine and Remove methods (called when you add or remove an event handler) perform some kind of reference counting to determine how many times equivalent delegates should be invoked.

A simple example of this behavior can be demonstrated by creating two EventHandler delegates pointing to the same handler method.  Add one of those delegates to the event, but remove the other…the handler will never be called when the event is raised since both delegates are equivalent.  However, if you add the same delegate multiple times to the event, then the handler will be called as many times as the delegate was added, and you must remove the delegate (or an equivalent delegate) at least the same number of times in order for the handler not to be called.

Here’s the Console app used to verify this behavior.

using System;
using System.ComponentModel;

internal class
Program
{
private static readonly object myEventKey = new object();

private readonly EventHandlerList eventHandlers = new EventHandlerList();

private event EventHandler MyEvent
{
add
{
eventHandlers.AddHandler(myEventKey, value);
}
remove
{
eventHandlers.RemoveHandler(myEventKey, value);
}
}

private static void Main(string[] args)
{
Program program = new Program();

Console.WriteLine("Adding two delegates with distinct identities pointing to the same method.");

EventHandler myEventHandler1 = MyEventHandler;
EventHandler myEventHandler2 = MyEventHandler;

program.MyEvent += myEventHandler1;
program.MyEvent += myEventHandler2;

Console.WriteLine("Raising MyEvent (expecting MyEvent to be called twice).");

program.OnMyEvent(EventArgs.Empty);

Console.WriteLine("Removing a new delegate pointing to the previously added methods.");

EventHandler myEventHandler3 = MyEventHandler;

program.MyEvent -= myEventHandler3;

Console.WriteLine("Raising MyEvent (expecting MyEvent to be called twice, once for myEventHandler1 and once for myEventHandler2).");

program.OnMyEvent(EventArgs.Empty);

Console.WriteLine("Removing myEventHandler1 from the event handler list.");

program.MyEvent -= myEventHandler1;

Console.WriteLine("Raising MyEvent (expecting MyEvent to be called once, for myEventHandler2.");

program.OnMyEvent(EventArgs.Empty);

Console.WriteLine("Creating a specific delegate that will be used in both add and remove.");

EventHandler myEventHandler = MySpecificEventHandler;

Console.WriteLine("Adding specific delegate to the handler list.");

program.MyEvent += myEventHandler;

Console.WriteLine("Raising MyEvent, (expecting MyEvent to be called twice, once for myEventHandler2 and once for myEventHandler.");

program.OnMyEvent(EventArgs.Empty);

Console.WriteLine("Removing specific delegate from the handler list.");

program.MyEvent -= myEventHandler;

Console.WriteLine("Raising MyEvent, (expecting MyEvent to be called once, for myEventHandler2 that has never been removed.");

program.OnMyEvent(EventArgs.Empty);

Console.WriteLine("Press the 'Enter' key to stop the server.");

Console.ReadLine();
}

private static void MyEventHandler(object sender, EventArgs e)
{
Console.WriteLine("MyEventHandler Called.");
}

private static void MySpecificEventHandler(object sender, EventArgs e)
{
Console.WriteLine("MySpecificEventHandler Called.");
}

private void OnMyEvent(EventArgs e)
{
EventHandler myEvent = (EventHandler)eventHandlers[myEventKey];
if (myEvent != null)
{
myEvent(this, e);
}
}
}




…and this is the output of the test above:


DelegateIdentityBehaviorTest


Cool.  I became sure of something today that previously I had only thought I knew.

Thursday, April 19, 2007

The Value of Consistency and Discoverability

Today I found myself taking the code download from a book I’m reading (Dependency Injection in .NET) and restructuring the solutions and sample projects that it contained according to my employer’s project structure standards and guidelines (which, incidentally, I helped to create and currently maintain).

Then I found myself wondering “Why am I doing this?” and “What value is added by this extra effort?”.  This code will never be part of an official project or effort that I’m working on.  I don’t intend to use this for anything other than my own self-education.  Then it finally dawned on me…

There is unimaginable value in consistency and discoverability – repeatedly achievable results.  What does a McDouble taste like if you get it from your local McDonald’s…or a Big Mac?  How is it dressed?  I’m betting that it is probably consistent with the same orders from my own local McDonald’s and that these locations have similar, if not identical, menu options…and therein lies a good portion of their value and appeal to such a wide consumer market.

In the code download, I wasn’t so much interested in the samples themselves (those are covered in the book), but in the accompanying unit tests (something that I still sometimes struggle with).  Placing these projects and solutions into a common structure allowed me to easily identify the tests accompanying the code and their relationship to the samples.

Enterprise Library 3.0 tests and Windows Vista

Well, I've downloaded, installed, built, and tried to run the tests for Enterprise Library 3.0. I am impressed that they have added perhaps 2000 or so new tests for the functionality. Unfortunately, in my first test run in VS, there were 146 failures. In the EntLib 2.0, I would typically get 2 or 3 failures as a result of timing issues (most notably when writing to the Event Log and determining number of entries written, or raising WMI events and turning around to see the number of events raised). Re-running just these tests made those issues go away.

I suspect that some of the problems I'm having getting the tests to pass go back to Windows Vista and UAC (User Account Control). I run as non-admin (as any good developer should) in my day-to-day activities, but even running these tests as a member of the Administrators group fails under Vista (yes, I know the group SID is not enabled by default). Finally, running as an 'elevated' Administrator allows most of the tests to pass (36 failures as of last run...investigating now). I want repeatable results.

Let say that I just expected more... Either a "To execute the tests as non-admin..." section in the documentation, or a "For Windows Vista users and administrators" section. Maybe even "the unit tests provided have the following dependencies" (cough). Perhaps the EntLib team (and Microsoft in general) should look up the definition of 'unit tests', as this has been sullied enough by Team System and is confusing (mis)information for novices looking for guidance.

Don't get me wrong, I love Microsoft. They have been, directly or indirectly, responsible for my paychecks for the past 15 years or so. They are learning; it's just...slowly.