This is a problem I recently ran into:

- I am using Visual Studio Test Suite 2008 to create load tests that run multiple unit tests

- I created unit tests that run by getting configuration settings from an app.config file

- If I run the unit test by itself, they pass

- I placed the unit tests in a load test and ran that: the load test fails at the line of code that tries to access the app.config:

if (System.Configuration.ConfigurationManager.AppSettings["xxx"].Equals("xxx"))

Fixing this is extremely simple. In fact, this blog post describes it perfectly: http://www.apexa.net/Blog/web_design_Blog_20080602.aspx.

View the properties of the load test's Run Settings and set the property: "Run unit tests in application domain" to TRUE.

Done!

I have always had a problem doing performance testing (aka load testing). I am quite adept at recording web tests and collecting counters, but I still find it troublesome interpreting the counters that I gather from load tests. The problem is that I need a plan that will produce relevant results with which I can conclude with confidence: "This system is a juggernaught! It can withstand more hits than Google.com." If you are like me and you have trouble planning your load tests, then maybe this post will help you. The following post describes how I like to run my own plan and what conclusions I can make with this plan.

Special thanks goes out to Jeff Dunmall, our co-CEO, whose article "Real-World Load Testing Tips to Avoid Bottlenecks When Your Web App Goes Live" got me to write something more about load testing here, and Randar Puust who uses this plan (and thus now I use it too).

There are a number of facets to load testing:

- setting up a load testing rig and setting up your environment in  Visual Studio

- recording the web tests

- planning the load tests

- collecting performance counters and figuring out what the recorded result mean

Unless you have a plan, planning the load tests and reading the results will be meaningless. A performance test plan will keep you on track throughout the last two phases mentioned above. The results from a test plan will allow you to determine:

- maximum number of users that can hit the site before it becomes unresponsive

- can the system recover from overload

- are there memory leaks in the code

A performance test plan consists of four types of tests that need to be run in the following order:

1. Performance Tests

2. Load Tests

3. Stress Tests

4. Endurance Tests

1. Performance Tests

These are the benchmark tests. By this point you should have recorded web tests which cover the most-used parts of your web application. Now, you will create a load test that gathers and runs all of the web tests that you have recorded. Run the load test with settings similar to these:

- load: low end of the expected number of users for the system. I like to start with 5 or 10.

- time: 10 minutes

- think time: 5 seconds

- warmup: 30 seconds to 1 minute is more than enough

A load test like this should not stress the web server, it should be low-impact. The goal of this test is for it to be run later on in this plan and be used to compare against (i.e. a benchmark). Collect and save the result counters from the web server and from the client. Web Server counters to collect are: "CPU %", "Used Memory". Client counters to collect are: "Respose time per page", "Number of Requests per second". Other counters you might like to save are "pages that take the longest time to load", "number of errors", "network usage" - these are useful for neat reports that you might want to compile.

2. Load Tests

The load tests are going to stress the web application and try and find its breaking point. The goal of these test will be to find the lucky number X. This number is the number of users that are concurrently hitting the system and that cause the system to be unresponsive or response times to be unacceptable. For this part of the plan you will need to run a number of load tests. In each case, increase the number of users and collect the counters:

- load: 10 users, 15 users, 20 users, 50 users.... until the response times get too large, or server CPU % gets dangerously high.

- time, think time, warmup: same as for the performance test. Basically we are running the performance test from step 1but with more and more users.

By the end of this step you will know what is the maximum number of users that can hit the system.

3. Stress Tests

Stress tests check that the web application can recover from errors induced by overload.

- load: X + 20% (where X is the magic number from step 2 which causes the system to be unstable)

- time: 1 minute

- think time: 5 seconds

After this test is run the system should become unresponsive. If it is not unresponsive, then increase the 20%. One way to check if it is unresponsive is to check that response times are suddenly exponentially far worse than the results in step 2 and CPU % usage on the server is close to 100%.

Immediately after this test runs run the performance test from step 1. If the performance test runs fine and the results are comparable to the test that was run at step 1, then your system passes! It can safely recover from overload issues.

4. Endurance Tests

The endurance tests are unrelated to the other 3 tests. These tests are run over a long period of time, which causes many users to hit the site. If there are memory leaks, then over the course of this endurance test the free memory on the web server should decrease dramatically.

- load: X - 20% (where X is the magic number from step 2 which causes the system to be unstable). We'll use -(minus) 20% because we don't want to overload the system

- time: 12 hours

- think time: 5 seconds

Remember to record the "Free" or "Used Memory" on the web server. The goal at the end of this step is to make sure that the memory usage on the web server has not decreased dramatically and thus there are no memory leaks in your code. If that is the case with your results then tip of the hat to you!

At the end of these four types of tests you should have a better idea of where your system stands in terms of load that it can handle. You will know:

- what is the maximum number of users it can handle before performance is unacceptable

- can it recover after an overload

- are there any memory leaks

Happy testing!

I wanted to contribute a little to the book discussions we've been having on the imason blogs. Scott had two of these [1] [2] and Steve had another.

I just finished reading Rich Dad Poor Dad by Robert Kiyosaki.

I'm not too much into self-help books. I always feel like they are just a way for the author to make some more money. This book is similar. It is written in very plain English and has a lot of repetition - it is meant for anybody and everybody to read. However, I am impressed with the motivation it provides. (Just as a self-help book should). Maybe that's why it was on the New York Times best seller's list. Similarly it has its bad sides. It is a self-help book and this article does a great job as showing exactly how the author is just making money off of this book - and I partly agree.

But, nevertheless, overall, it offers great motivational advice, such as:
1. Act rather than not act. If you don't act, you will never succeed
2. Take risks sometimes, don't be a chiken. This links to #1 I guess
3. Surround yourself with (e.g. hire) people smarter than you
4. Learn a market by reading books or going to courses
and always:
5. "Think and Make Money" not "Work Hard and Make Money". - i.e. always try to think of new ways to invest your dollar

The financial advice is:
1. Learn accounting (so you can know what an asset/liability/income/expense is)
2. Learn investing (when to know what is a good deal)
3. Learn marketing (so you can notice trends in advance)
4. Learn the law for your market (so you can take advantage of it)

Then the master rule: buying assets that make you monthly income rather than buying liabilities that cause you monthly expenses.

That's a short version of it, except with a lot more motivation and "get out there and stop being so lazy", and "you can be rich too" talk.

Seeing how Steve is doing a good job of advertising his mo, I thought I would do the same... to keep the competition alive!

Here is how my mo is measuring up:

Does it look vaguely familiar?

If you try to programaticaly set/remove values of fields in InfoPath, you might have come across the following error message:
"Schema validation found non-data type errors"

A number of blogs and MSDN articles describe what to do in this case. They work for setting values, however I could not find how to clear values.

• Dooke's SharePoint Place has a great post about it, in fact this post solved my problem with a small change on my part
  http://www.sharepointblogs.com/dooke/default.aspx
• The InfoPath team blog also talks about it, with similar code
  http://blogs.msdn.com/infopath/archive/2006/11/28/the-xsi-nil-attribute.aspx
• The Microsoft Support page is also useful
  http://support.microsoft.com/kb/826998

The problem

Values that are non-string (DateTime, Time, Date, Boolean, Whole Number, Decimal) cannot be blank (Empty String). If a value is set to the emptry string you will get the error above, because the empty string is considered a string and you are trying to set it to a field that is non-string. When one of these non-string fields is blank, it has an attribute "xsi:nil" that is set to "true". As soon as you type a value in this field in InfoPath the attribute is cleared.

The solution

Therefore, if you are trying to set values programatically you need to clear the "xsi:nil" attribute, or set it to "false". And if you are clearing the value of such a field, then you need to place the "xsi:nil" attribute back in, or set it to "true".

Other blogs on the internet say that clearing the value and setting the xsi:nil attribute to true is very simple, however the snippets of code they offer do not work.

• This is because a nil field looks like so: <my:value />
• After you type something, it looks like so: <my:value>123</my:value>
• If you try and add the nil attribute in <my:value></my:value> you will get the "Schema validation found non-data type errors" error when you load the form.

The solution is to follow the steps below, depending on what you need to do with your field.

Editing a non-string field:

• If you are going to try and programatically edit a field that is non-string, then you have to remove its "nil" attribute
• Now you can change its value
  

Code (as on all other blogs and articles, this piece was taken from http://blogs.msdn.com/infopath/archive/2006/11/28/the-xsi-nil-attribute.aspx):

//Create a Navigator object for the main data source

XPathNavigator xn = this.MainDataSource.CreateNavigator();

 

//Create a navigator object for the field (node)

//where we want to set the current date value

XPathNavigator xnfield1 = xn.SelectSingleNode("/my:myFields/my:field1", this.NamespaceManager);

 

//Check if the "nil" attribute exists on this node

DeleteNil(xnfield1);

 

//Create a new dateTime object for the current date

DateTime curDate = new DateTime(DateTime.Today.Year, DateTime.Today.Month, DateTime.Today.Day);

 

//Set the value of field1 to the current date in the

//correct format: yyyy-mm-dd

xnfield1.SetValue(curDate.GetDateTimeFormats().GetValue(5).ToString());

public void DeleteNil(XPathNavigator node)
{

if (node.MoveToAttribute("nil", http://www.w3.org/2001/XMLSchema-instance))
      node.DeleteSelf();

}

Clearing a non-string field:

• If you are going to programatically erase the value of a non-string field, then you have to re-construct the XML node to what InfoPath expects and add the "nil" attribute. This code is slightly different from dooke's Sharepoint place blog.

Code (slight change from dooke's code):

public static void InsertNil(XPathNavigator node)

{

if (!node.MoveToAttribute("nil", "http://www.w3.org/2001/XMLSchema-instance"))

      {

            string result = string.Empty;

            int endIndex = node.OuterXml.IndexOf(">");

            result = node.OuterXml.Substring(0, endIndex) + " xsi:nil=\"true\" />";

            node.OuterXml = result;

}

}

This code will reconstruct the <my:value>123</my:value> to <my:value xsi:nil="true" /> which will clear the field.

In my particular case, I wanted to clear the field, no matter what it was (string or non-string):

private void ClearAllFields()

{

XPathNavigator node = xNav.SelectSingleNode(xPath, xNameSpace);

      if (node == null)

            return;

      try

      {

            // try this as a Date, Integer, Boolean node, set the nil value

            InsertNil(node);

      }

      catch

{

            // if it is a string, the nil value won't work, so just set it to empty

            node.SetValue(string.Empty);

      }

}

{

if (!node.MoveToAttribute("nil", "http://www.w3.org/2001/XMLSchema-instance"))

      {

            string result = string.Empty;

            int endIndex = node.OuterXml.IndexOf(">");

            result = node.OuterXml.Substring(0, endIndex) + " xsi:nil=\"true\" />";

            node.OuterXml = result;

}

}