Feeds:
Posts
Comments

Posts Tagged ‘c#’

Blog has moved! You should check this post out on kevinpelgrims.com

Now that we’ve covered the Task Parallel Library, it’s time to move on.

What is PLINQ?

PLINQ stands for Parallel LINQ and is simply the parallel version of LINQ to Objects. Just like LINQ you can use it on any IEnumerable and there’s also deferred execution. Using PLINQ is even easier than using the Task Parallel Library!

Regular for loop and LINQ compared to PLINQ (with time in seconds)

How do we use PLINQ?

You can even make existing LINQ queries parallel simply by adding the AsParallel() method. That’s how easy it is! This makes it easy to use the power of parallelization, while enjoying the readability of LINQ. Isn’t that great?

var employees = GetEmployees();

// Regular LINQ
var query = employees.Select(e => e.Skills.Contains("C#"));

// Extension method style PLINQ
var queryParallel1 = employees.AsParallel()
                              .Select(e => e.Skills.Contains("C#"));

// Query expression style PLINQ
var queryParallel2 = from e in employees.AsParallel()
                     where e.Skills.Contains("C#")
                     select e;

Important fact: PLINQ uses all the processors of your system by default, with a maximum of 64. In some cases you might want to limit this, to give a machine some more power to take care of other tasks. Everybody deserves some CPU time! So don’t be greedy and use WithDegreeOfParallelism() on heavy queries. Following example uses a maximum of 3 processors, even if there are 16 available.

var queryDegree = employees.AsParallel()
                           .WithDegreeOfParallelism(3)
                           .Select(e => e.Skills.Contains("C#"));

By default PLINQ doesn’t care about the order of your output, compared to the input. This is because order preservation costs more time. You can enable order preservation though, again in a very simple way, by using the AsOrdered() method. It’s good to know that OrderBy() will also take care of order preservation.

var employees = GetEmployeesOrderedByName();

var queryOrdered = employees.AsParallel()
                            .Select(e => e.Skills.Contains("C#"))
                            .AsOrdered();

We want more!

PLINQ has a lot more to offer than what we talked about here, so be sure to use Google and MSDN if you want to know more. Check out this “old” (2007) yet interesting article on PLINQ from MSDN magazine. An important read is Understanding Speedup in PLINQ on MSDN, which explains a bit more of how PLINQ works and why it sometimes defaults to sequential mode anyway.

Advertisements

Read Full Post »

Blog has moved! You should check this post out on kevinpelgrims.com

I have talked about parallel programming in .NET before, very briefly: Parallel programming in .NET – Introduction. This follow-up post is long overdue 🙂

What is the TPL?

The Task Parallel Library is a set of APIs present in the System.Threading and System.Threading.Tasks namespaces. The point of these APIs is to make parallel programming easier to read and code. The library exposes the Parallel.For and Parallel.ForEach methods to enable parallel execution of loops and takes care of spawning and terminating threads, as well as scaling to multiple processors.

How do we use the TPL?

Following code uses the sequential and the parallel approach to go over a for-loop with some heavy calculations. I use the StopWatch class to compare the results in a command window.

//Sequential
watch = new Stopwatch();
watch.Start();
for (int i = 0; i < 20000; i++)
{
    SomeHeavyCalculations(i);
}
watch.Stop();
Console.WriteLine("Sequential Time: " + watch.Elapsed.Seconds.ToString());

//Parallel
watch = new Stopwatch();
watch.Start();
System.Threading.Tasks.Parallel.For(0, 20000, i =>
{
    SomeHeavyCalculations(i);
}
);
watch.Stop();
Console.WriteLine("Parallel Time: " + watch.Elapsed.Seconds.ToString());

The result of running this on my laptop (with multiple cores) looks like this:

Result of comparison sequential - parallel

As you can see, the parallel for-loop runs A LOT faster than the sequential version. By using all the available processing power, we can speed up loops significantly!

Below is a screenshot of the task manager keeping track of what’s happening  while executing the sequential and the parallel. What we can see here is that at first (where the red arrow is pointing at) we only use 1 core heavily. When the parallel code kicks in, all cores peak.

Task manager during comparison sequential - parallel

So, looking at the above code, implementing all this parallelism doesn’t seem to be that hard. The TPL makes it pretty easy to make use of all the processors in a machine.

Creating and running tasks

It’s possible to run a task implicitly by using the Parallel.Invoke method.

Parallel.Invoke(() => DoSomething())
Parallel.Invoke(() => DoSomething(), () => DoSomethingElse())

All you need to do is pass in a delegate, using lamba expressions makes this easy. You can call a named method or have some inline code. If you want to start more tasks concurrently, you can just insert more delegates to the same Parallel.Invoke method.

If you want more control over what’s happening, you’ll need to use a Task object, though. The task object has some interesting methods and properties that we can use to control the flow of our parallel code.

It is possible to use new Task() to create a new task object, but it’s a best practice to use the task factory. (Note that you can’t use the task factory if you want to separate the creation and the scheduling of the task.)

// Create a task and start it
var task1 = new Task(() => Console.WriteLine("Task1 says hi!"));
task1.Start();

// Create a task using the task factory
var task1 = Task.Factory.StartNew(() => Console.WriteLine("Task1 says hi!"));

You can also get results from a task, by accessing the Result property. If you access it before the task is completed, the thread will be blocked until the result is available.

Task<int> taskreturn = Task.Factory.StartNew(() =>
  {
    int calc = 3 + 3;
    return calc;
  });
int result = taskreturn.Result;

To be continued..

You can chain tasks by using the Task.ContinueWith method. It’s also possible to access the result of the preceding task in the next one, using the Result property.

// Regular continuation
Task<int> task1 = Task.Factory.StartNew(() => 5);
Task<string> task2 = task1.ContinueWith(x => PrintInt(x.Result));

// Chained continuation
Task<string> task1 = Task.Factory.StartNew(() => 5)
                     .ContinueWith(x => PrintInt(x.Result));

The methods ContinueWhenAll() and ContinueWhenAny() make it possible to continue from multiple tasks by taking in an array of tasks to wait on and the action to be undertaken when those have finished. More about those functions can be found on MSDN.

The force is strong with this one

We only looked at a few functions of the TPL and I think it’s clear this is a very powerful library. When working on applications that need a lot of processing power, parallel programming in .NET can make it easier to improve performance, a lot.

Resources

Of course there is a lot more to TPL than covered in this small introduction, so go ahead and explore!

Read Full Post »

Blog has moved! You should check this post out on kevinpelgrims.com

Yesterday I attended a session on unit testing by Roy Osherove in Copenhagen. As I am trying to learn more about unit testing and TDD by applying it in a pet project, it was very interesting to see what a veteran like Roy had to say about the subject of unit testing. Also very interesting was his approach in this session, as he tried to teach us about good habits by showing us bad (real world) examples.

He also pointed out that anyone interested in writing unit tests and working test driven should do test reviews. It can be used as a learning tool, for example test review some open source projects. But it can also be used internally almost as a replacement of code reviews, because reviewing tests takes a lot less time and should give you a good idea of what the code is supposed to do (when working test driven).

I took some notes during the session that I would like to share – and keep here for my own reference 😉 I wrote down most of his tips, so to the unit testing experts out there some of it might seem really basic. But I thought it was interesting to have it all written down.

Three important words

The basic, yet very important requirements for tests:

  • Readable
  • Maintainable
  • Trustworthy

Unit test VS integration test

Unit tests are used for testing stuff in memory. The tests don’t change and they’re static. They don’t depend on other things.

Integration tests would be used when there is a dependency on the filesystem, a database, a Sharepoint server, etc.

Unit tests and integration tests have their own testproject!

Basics

  • Avoid test logic: too complicated
    • Ifs, switches, for loops, ..
  • No multiple asserts
    • This can be okay when you’re asserting using the same object
  • Avoid “magic numbers”
    • Using the number 42 somewhere raises the question whether it is important that the number is equal to 42; a good idea would be to use a variable with a descriptive name
  • Don’t assert on calculations or concatenations
    • Assert(“user,password”, Bleh()) is better than Assert(user + “,” + password, Bleh())
  • Don’t change or remove tests!
  • DateTime.Now (or friends like Random) –> NOT okay! These values change everytime
  • Test only publics

Reuse

  • Factory methods (usually in the same class as the tests using them)
    • make_xx
  • Configure initial state
    • init_xx
  • Common tests in common methods
    • verify_xx

Tests are isolated

  • Don’t call other tests in a test
  • No shared state, have cleanup code for shared objects

Mock != Stub (in short)

  • Mock = used for asserts
  • Stub = used to help the test
  • Fake = can be both

Tip

If you need to test things related to a database, that would be an integration test and it’s a good idea to use the TransactionScope class in .NET so you can rollback everything when the test is done.

Read Full Post »

Bing Maps – Geocoding and Imagery

Blog has moved! You should check this post out on kevinpelgrims.com

Since I did a lot of GIS related stuff recently for work, I decided I’d have some fun with the Bing Maps API. I’ve been using Bing to display maps as a base for multiple layers of data in combination with MapGuide OS and needed to convert addresses to geocodes (coordinates). Afterwards I decided to play with it some more and I created a little app in C# that makes more use of Bing Maps.

If you want to get started with the Bing API, you’ll first need to get a key. More info on that can be found on MSDN.

Alright, now you got your key, let’s get started!

First thing you need to do to use the Bing Maps service, is adding a service reference. We’ll start out with some geocoding, so we need the geocode service. The addresses of the available services can be found here.

Click "Add Service Reference..." to open the service reference dialog

The we can use the following code to “geocode” an address (get the coordinates):

private String Geocode(string address)
{
    GeocodeRequest geocodeRequest = new GeocodeRequest();

    // Set credentials using a Bing Maps key
    geocodeRequest.Credentials = new GeocodeService.Credentials();
    geocodeRequest.Credentials.ApplicationId = key;

    // Set the address
    geocodeRequest.Query = address;

    // Make the geocode request
    GeocodeServiceClient geocodeService = new GeocodeServiceClient("BasicHttpBinding_IGeocodeService");
    GeocodeResponse geocodeResponse = geocodeService.Geocode(geocodeRequest);

    return GetGeocodeResults(geocodeResponse);
}

GetGeocodeResults is just a function that I made to print out the response on the screen. As seen here:

Get the coordinates for an address

(There are some extra options available. You could, for example, tell the service to only return “high confidence” results. But I’m not going to talk about that here.)

Because getting the coordinates was so easy, I decided to also implement reverse geocoding. Which is (as you would expect) converting coordinates to an address.

private String ReverseGeocode(double latitude, double longitude)
{
    ReverseGeocodeRequest reverseGeocodeRequest = new ReverseGeocodeRequest();

    // Set credentials using a Bing Maps key
    reverseGeocodeRequest.Credentials = new GeocodeService.Credentials();
    reverseGeocodeRequest.Credentials.ApplicationId = key;

    // Set the coordinates
    reverseGeocodeRequest.Location = new BingMapsSoap.GeocodeService.GeocodeLocation() { Latitude = latitude, Longitude = longitude };

    // Make the reverse geocode request
    GeocodeServiceClient geocodeService = new GeocodeServiceClient("BasicHttpBinding_IGeocodeService");
    GeocodeResponse geocodeResponse = geocodeService.ReverseGeocode(reverseGeocodeRequest);

    return GetGeocodeResults(geocodeResponse);
}

Converting coordinates to addresses is easy!

After getting this far in about 15 minutes of figuring it out and coding, I couldn’t stop there! I decided to add some basic imagery for the address/coordinates that are converted. For imagery you need to add a reference to the imagery service first. Writing code for this is also pretty easy, as there are plenty of examples on MSDN that can be useful. It seems Microsoft really put some effort into  documenting this right 🙂

private void GetImagery(double latitude, double longitude)
{
    MapUriRequest mapUriRequest = new MapUriRequest();

    // Set credentials using Bing Maps key
    mapUriRequest.Credentials = new ImageryService.Credentials();
    mapUriRequest.Credentials.ApplicationId = key;

    // Set the location of the image
    mapUriRequest.Center = new ImageryService.Location();
    mapUriRequest.Center.Latitude = latitude;
    mapUriRequest.Center.Longitude = longitude;

    // Set map style and zoom level
    MapUriOptions mapUriOptions = new MapUriOptions();
    mapUriOptions.Style = MapStyle.AerialWithLabels;
    mapUriOptions.ZoomLevel = 17;

    // Set size of the image to match the size of the image control
    mapUriOptions.ImageSize = new ImageryService.SizeOfint();
    mapUriOptions.ImageSize.Height = 160;
    mapUriOptions.ImageSize.Width = 160;

    mapUriRequest.Options = mapUriOptions;

    ImageryServiceClient imageryService = new ImageryServiceClient("BasicHttpBinding_IImageryService");
    MapUriResponse mapUriResponse = imageryService.GetMapUri(mapUriRequest);
    BitmapImage bmpImg = new BitmapImage(new Uri(mapUriResponse.Uri));
    bingImage.Source = bmpImg;
}

That code gives us this result:

Looks pretty good for a small app that took almost no time to make. The Bing Maps API is pretty straight-forward to work with and MSDN has some good samples to get started. So if you’re interested in working with Bing Maps, be sure to check out the documentation.

Now go and have fun with this!

Read Full Post »

PowerShell Script Commander

Blog has moved! You should check this post out on kevinpelgrims.com

I wrote this entry last year September, but for several reasons the post and the code never made it to the interwebs. After running into it again I decided to post it anyway and upload the code to Codeplex. So, here goes!

A while back I created this PowerShell script to remotely manage services (see Remote Service Manager), a bit later I made the available help a bit more professional (see Add help to your own PowerShell scripts). Now I was thinking it would be cool to create an interface in .NET for this script. So I started developing a WinForm application around the script. Initially, it looked like this:

This was the first (working) prototype of the RSM-GUI. But then I started thinking it would be much cooler to create an application that can run all scripts, even those with parameters. I started thinking about how to do this and actually it didn’t seem so hard. I started by creating a form that can execute simple PowerShell commands and show the output, then added support for running scripts stored on the hard drive. For the parameters I just made sure they are all displayed in a grid, where you can give them a value by simply typing something in. And that was it! I now have a Windows application that can run scripts on a more visual way. Now you can simply load the script with an OpenFileDialog and see what parameters are available. If you still don’t fully understand how the script works, you can press the Get-Help button to display the help information in the outputbox.

And now, it looks like this:

PowerShell Script Commander help function

PowerShell Script Commander running the RSM script, with parameters

A lot of work needs to be done to make this more reliable and to make it work with all scripts on the planet 😉 But I think it’s already pretty cool.

The source code can be found here: http://pscommander.codeplex.com/

Read Full Post »

Blog is moving! You should check this post out on kevinpelgrims.com

Autodesk Infrastructure Map Server logoI encountered a problem today while working on a website that uses Autodesk Infrastructure Map Server 2012 (aka MapGuide) and it’s API. To programmatically create a new layer on a map it seems to be recommended to use the LayerDefinitionFactory that is provided with the installation of the server. Now the problem is that this factory is only provided in PHP and I am working in C#. I could install PHP on the server and try to get all freaky with .NET-PHP communication. But it seemed easier to just port the factory to C# (it’s not that big anyway).

I wrote some basic functions and then only converted the stuff I need for this particular project, but it should be very easy to convert the other functions when you need them. I did not rewrite the sprintf() function that can be found in PHP, because it is a lot easier to use String.Format() if you adjust the template files. So, this is what one of the templates now look like after changing all the occurrences of %s to the C# version with {0}, {1}, etc.:

<?xml version="1.0" encoding="UTF-8"?>
<LayerDefinition xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="LayerDefinition-2.3.0.xsd" version="2.3.0">
  <VectorLayerDefinition>
    <ResourceId>{0}</ResourceId>
    <FeatureName>{1}</FeatureName>
    <FeatureNameType>FeatureClass</FeatureNameType>
    <Geometry>{2}</Geometry>
    {3}
  </VectorLayerDefinition>
</LayerDefinition>

And here is the code:

using System;
using System.Text;
using System.IO;

/// <summary>
/// Factory for MapGuide layer definitions.
/// </summary>
public class LayerDefinitionFactory
{
	string templatePath;

	public LayerDefinitionFactory(string path)
	{
		templatePath = String.Concat(path, "{0}.templ");
	}

	/// <summary>
	/// Read the template required for the layer definition.
	/// </summary>
	/// <returns>Returns (a part of) the layer definition in XML.</returns>
	private string ReadTemplate(string filename)
	{
		StreamReader reader = new StreamReader(String.Format(templatePath, filename));
		string templateDefinition = reader.ReadToEnd();
		reader.Close();
		return templateDefinition;
	}

	/// <summary>
	/// Create a new layerdefinition.
	/// </summary>
	public string CreateLayerDefinition(string resourceId, string featureClass, string geometry, string featureClassRange)
	{
		string[] values = new string[] { resourceId, featureClass, geometry, featureClassRange };
		return String.Format(ReadTemplate("layerdefinition"), values);
	}

	/// <summary>
	/// Create a new line rule for the layer definition.
	/// </summary>
	public string CreateLineRule(string lineStyle, string thickness, string color)
	{
		string[] values = new string[] { lineStyle, thickness, color };
		return String.Format(ReadTemplate("linerule"), values);
	}

	/// <summary>
	/// Create the line type style with line rules for the layer definition.
	/// </summary>
	public string CreateLineTypeStyle(string lineRules)
	{
		string[] values = new string[] { lineRules };
		return String.Format(ReadTemplate("linetypestyle"), values);
	}

	/// <summary>
	/// Create the scale range for the layer definition.
	/// </summary>
	public string CreateScaleRange(string minScale, string maxScale, string typeStyle)
	{
		string[] values = new string[] { minScale, maxScale, typeStyle };
		return String.Format(ReadTemplate("scalerange"), values);
	}
}

Read Full Post »

Blog is moving! You should check this post out on kevinpelgrims.com

Testing e-mails sent from .NET code

It’s always a big hassle to test sending e-mails from websites or applications written in .NET (or any other platform/language). Your inbox gets flooded or even worse, customers’ inboxes get flooded. Or you’ve set up a dummy account to test if e-mails get sent. But sometimes e-mail addresses need to be picked up dynamically depending on certain parameters, from a database, for example. Testing gets annoying here. You can debug and check if the address you want is the right one, you can bother people to check their inbox everytime you’re testing, ..

There has to be a better solution, right? Yes there is!

Developers, let’s start testing!

Thanks to this little tool I found last week, this has become a lot easier. It’s actually been around for about 2,5 years already. Too bad I never found out about it earlier 🙂

UPDATE: Just found out about this thing, which is on CodePlex, so we can play with the code! It’s called smtp4dev and can be found here: http://smtp4dev.codeplex.com/.

Here are URLs to the 3 blogposts by Donovan Brown about the little server app:

Note: Don’t download the application from the first blogpost! The post about POP3 has the latest version.

Be sure to read the documentation, it has everything you need to know about using this tool.

Too bad Donovan has never put this on Codeplex (or any other open source repository), because I think we can all imagine some nice features to add to this little useful thing..

Read Full Post »

Older Posts »