Sunday, September 21, 2008

How to deploy Silverlight 2 Beta 2 from CruiseControl.Net to IIS using MSBuild?

Problem

Deploying Silverlight 2 Beta 2 using MSBuild with CruiseControl.Net in Windows 2003 64bit is bit tricky.

Assumption

1) Familiar with MSBuild
2) Familiar with IIS 6
3) Using .Net 3.5 SP1
4) VS 2008

Step by step instruction

1) First error from CruiseControl.net was

(GetFrameworkPaths target) ->
2008-09-20 22:02:06,368 [Moo:DEBUG] C:\Program Files\MSBuild\Microsoft\Silverlight\v2.0\Microsoft.Silverlight.Common.targets(78,9): error : The Silverlight 2 SDK is not installed.

To Fix this just install Microsoft® Silverlight™ 2 Software Development Kit Beta 2

2) Use <Platform>AnyCPU</Platform> to compile

3) In CruiseControl.Net use %systemroot%\Microsoft.NET\Framework\v3.5\msbuild.exe to build.

4) In 64 bit, xap file does not get copied to Web Application project's ClientBin. So make sure to copy compiled xap file from Silverlight project's bin as shown below.
Configuration can be either Debug or Release build.


<Target

    Name="Silverlight-Hack">

 

    <PropertyGroup>

      <XapFile>$(AppRoot)Moo.Silverlight\bin\$(Configuration)\Moo.Silverlight.xap</XapFile>

      <CopyXapTo>$(AppRoot)Moo.Web\ClientBin\</CopyXapTo>

    </PropertyGroup>

 

    <MakeDir Directories="$(CopyXapTo)" Condition=" !Exists('$(CopyXapTo)') " />

 

    <Copy SourceFiles="$(XapFile)" DestinationFolder="$(CopyXapTo)" />

  </Target>



5) Make sure to add Silverlight 2 mime types in IIS

.xaml application/xaml+xml
.xap application/x-silverlight-app

6) Make sure add ClientAccessPolicy.xml to WCF service project if the Silverlight will be accessing the service from different application domain.
For more information using ClientAccessPolicy.xml


<?xml version="1.0" encoding="utf-8"?>

<access-policy>

  <cross-domain-access>

    <policy>

      <allow-from http-request-headers="*">

        <domain uri="*"/>

      </allow-from>

      <grant-to>

        <resource path="/" include-subpaths="true"/>

      </grant-to>

    </policy>

  </cross-domain-access>

</access-policy>



Conclusion

Hopely, when Silverlight 2 is released some of the bugs mentioned above will be addressed.

Saturday, September 20, 2008

How to create load generator for simulating concurent hits?

Problem
We had to quickly generate some load on BizTalk and we needed some quick poorman's load generator to do it. Also this load generator was required to process requests in batch.

Here is the requirements:
1) Generate load simulating x number of concurrent connections to WCF net.tcp hosted by BizTalk.
2) There is limited resources(memory) on the load generating machines.
3) Must maintain x number of concurrent connections.
4) Must simulate scale out.

Explanation of above requirements:
1) In BizTalk if receive port is cofigured to use WCF net.tcp there is setting to control concurrent connection and from client in WCF netTcpBinding maxConnections is used to configure the setting as shown below (line 65):


   60 <netTcpBinding>

   61   <binding name="Default_NetTcpBinding" closeTimeout="00:05:00"

   62     openTimeout="00:05:00" receiveTimeout="00:10:00" sendTimeout="00:10:00"

   63     transactionFlow="false" transferMode="Buffered" transactionProtocol="OleTransactions"

   64     hostNameComparisonMode="StrongWildcard" listenBacklog="10"

   65     maxBufferPoolSize="524288" maxBufferSize="65536" maxConnections="10"

   66     maxReceivedMessageSize="65536">

   67     <readerQuotas maxDepth="32" maxStringContentLength="8192" maxArrayLength="16384"

   68       maxBytesPerRead="4096" maxNameTableCharCount="16384" />

   69     <reliableSession ordered="true" inactivityTimeout="00:10:00"

   70       enabled="false" />

   71     <security mode="None">

   72       <transport clientCredentialType="Windows" protectionLevel="EncryptAndSign" />

   73       <message clientCredentialType="Windows" />

   74     </security>

   75   </binding>

   76 </netTcpBinding>



2) For this poorman's generator ThreadPool.QueueUserWorkItem was used. The more threads were queued, more the application would consume memory and there was need for some threadshhold. There were many ways to achieve what we wanted, using Semaphore, BackgroundWorker, and publish and subscriber.

3) Given set period of time and given set of batch of request, we had to constantly push x number of messages to BizTalk.

4) Simulate some kind of scale out approach. So we need to be able to spawn multiple threads of this load generator.

References used to research and learn

I like using the resource below for quick reference.
My favorite resource

Assumption

1) This article will not focus on performance of thread pooling. It is simply load generator.

About the project

Download: PoormansLoadGenerator.zip

1) TestService.cs was used to wrap LoadGenerator.cs implementation. It was used for spawning the worker thread to simulate scaling out.

2) This simple generator was able to generate enough load on BizTalk and SQL server to bring CPU usage up to 80% performing 600 messages per second load. This number really depends on many variables like what BizTalk orchestration is doing, what messages, what stored procs are we executing, ect. For us this was the result we wanted using this load generator.

Requirements

.Net 3.5
VS2008

Step by step instruction

Let's first look at Load Generator's entry point.


class Program

{

    static void Main(string[] args)

    {

        IList<TestService> servicesToRun = new List<TestService>();

        for (int i = 0; i < Config.MaxLoadGenerators; i++)

        {

            LoadGenerator loadGenerator = new LoadGenerator();

            servicesToRun.Add(new TestService(loadGenerator));

        }

 

        Console.WriteLine("Starting the service...");

        foreach (TestService service in servicesToRun)

        {

            service.Start();

        }

 

        // Wait until key is pressed....

        Console.ReadLine();

 

 

        Console.WriteLine("Exiting the service...");

        foreach (TestService service in servicesToRun)

        {

            service.Shutdown();

        }

    }

}



Notice that Config.MaxLoadGenerators is used to control how many of instances of LoadGenerator can be running simultaneously. This is to control scaling out requirements on the single machine.


public class LoadGenerator :IService

{

    private static bool _shutDown;

 

    #region IService Members

 

    public void RegisterService()

    {

        _shutDown = false;

        ThreadPool.SetMaxThreads(Config.MaxConcurrentThread, Config.MaxConcurrentThread);

        ThreadPool.SetMinThreads(Config.MaxConcurrentThread, Config.MaxConcurrentThread);

    }

 

    public void Start()

    {

        int j = 0;

        long memory = GC.GetTotalMemory(true);

        long maxMem = Config.MaxMemoryToUse + memory;

 

        while (!_shutDown)

        {

            System.Threading.Thread.Sleep(Config.LoopWaitTime);

            memory = GC.GetTotalMemory(true);

            if (memory < maxMem)

            {

                for (int i = 0; i < Config.BatchSize; i++)

                {

                    string x;

                    string z;

                    SetData(out x, out z, i, j);

                    ThreadPool.QueueUserWorkItem(delegate(object notUsed)

                                {

                                    DoSomething(x, z);

                                });

                }

                ++j;

            }

        }

    }

 

    public void Shutdown()

    {

        _shutDown = true;

    }

 

    private void SetData(out string x, out string z, int i, int j)

    {

        x = "for = " + i;

        z = "while = " + j;

    }

 

    private static void DoSomething(string forStr, string whileStr)

    {

        // Put your implementations here

        System.Threading.Thread.Sleep(Config.DoSomethingSleepTime);

        Console.WriteLine(string.Format("{0}, {1}", forStr, whileStr));

 

    }

 

    #endregion

}



ThreadPool.SetMaxThreads is used to control maximum concurrent thread running at one point. ThreadPool.SetMinThreads is used so that it is always waiting to provide thread.

We were running things in batch mode and constantly pulling records from the SQL server only if the memory was avaiable and queing up the records queried from SQL server. Also, in order to maintain constant concurrent connections feeding BizTalk with messages we need to queue up the request so that as soon as the thread finishes its job there is thread available to be used.


public sealed class Config

    {

        public static int LoopWaitTime;

        public static int MaxConcurrentThread;

        public static long MaxMemoryToUse;

        public static int DoSomethingSleepTime;

        public static int MaxLoadGenerators;

        public static int BatchSize;

 

        private Config()

        {

        }

 

        static Config()

        {

            LoopWaitTime = 10;

            MaxConcurrentThread = 20;

            MaxMemoryToUse = 100000000; // 1 GB

            DoSomethingSleepTime = 500;

            MaxLoadGenerators = 2;

            BatchSize = 100;

        }

    }



There is enough configuration here to tweak the behavior of the load generator and by tweaking these configuration we were able to generate some big load (600 messages per seconds) while keeping the load generator simple and stable.

Conclusion

I think there are enough of commercial load generator including VS2008 Tester version simulating load. For quick and dirty this load generator is more than enough in my opinion.

Monday, September 1, 2008

Choosing ThoughtWorks Mingle, Version One, Rally, or Team Foundation Server 2008

Problem

We needed very good project management tools for managing and facilitating the development process. Also, for this review keep in mind that we take into account our budget which is very low.

It came down to 4 products for the review: ThoughtWorks Mingle, Version One, Rally, and Visual Studio Team System 2008 Team Foundation Server Workgroup Edition.

References used to research and learn

ThoughtWorks Mingle
Version One
Rally
Visual Studio Team System 2008 Team Foundation Server Workgroup Edition

Assumption

Readers understand the need for project management tools in the complex projects.

About the review

We installed and spent few days trying out the products on our dedicated Windows 2003 server hosted at 1&1 hosting - Enterprise I server.

Requirements

This was our requirements for choosing a right product for our needs:

1) Must have web access.
2) Must link to source controls.
3) Affordable.
4) Easy to use.
5) Be able to manage multiple products (small and big) from multiple clients.

Review

0) Saas vs On Site installation

Rally and VersionOne are very similar products in that they offer SaaS or software installed on site. Our strong preference is on on site installation so we can control everything about the product so SaaS model was not so appealing to us. Also, it was cool factor to have use our own sub domain New Age Solution Project Server and made much more sense when dealing with our customers.

1) Must have web access

All products offered some kind of web access. VersionOne lacked 64 bit support which forced our server to convert to 32 bit IIS 6. Not an ideal since we wanted to take advantage of 64 bit server's IIS. Mingle also ran as 32 bit process and we hope in future ThoughtWorks will release 64 bit. Mingle was somewhat slow in performance than other products.

2) Must link to source controls

They all linked to SVN which I prefered. For TFS it used Microsoft SQL server. We had some problem of all the sources being saved into SQL server. We prefered SVN which can be hosted else where with minimal server requirements not worrying about the database.

3) Affordable

All the products offered up to 5 users free or first year free and free is good :)
For Rally and VersionOne free versions had some functional limitation. For Mingle and TFS there were no limitation in function which made it very useful. As far as price went TFS was just way too expensive to use in small business. Microsoft partner program can help save some money but TFS could get very expensive over time. Rally, VersionOne, and Mingle are very affordable and does not require Visual Stuido Team edition.

4) Easy to use

This is where we have to say Mingle shines compare to other products. We have to say it was easy and customizable for our purpose. Only thing about Mingle is that it is slow. May be when it is released as 64 bit and ThoughtWorks optimized the app for next release Mingle might speed up.

5) Be able to manage multiple products (small and big) from multiple clients

Only Mingle and TFS offered this feature for free version. TFS has seemless integration to Visual Studio to checkout based on work item and be able to track work. With Mingle it can be done using SVN where during the commit process of SVN check in with specific comment which then Mingle will read the specifically formatted comment. It is just matter of training the developers.

Conclusion

We choose Mingle: see New Age Solution Project Server because of following things after going through our review criteria:

1) Web based so it can be accessed from any where.
2) Free up to 5 users fully functional.
3) Supports multiple projects for multiple clients. This is must feature we need since we have various clients constantly requesting features, fixes, or updates. We need a way to delegate this to developers. We can customize Mingle for this specific purpose very easily.
4) Easy to use. It took 30 minutes to install and setup and a day of playing around to understand Mingle's potential. It took another day of playing to make it work for complex project.
5) There is potential to hook into JIRA, BugZilla, CruiseControl or any other ThoughtWorks products.