September 2007 - Posts

Everyone knows UAC by now. One core element of Vista's UAC vision is the elevation of processes, used to run something as an administrator. Typically, you open up a command line instance (cmd.exe) which has been elevated already prior to invoking commands that require elevation. But what if you're already in a non-elevated command prompt window and need to start an executable that runs elevated? In a such a scenario a tool like the one shown below might be useful:

using System;
using System.Diagnostics;
using System.Reflection;

class Program
{
    static void Main(string[] args)
    {
        if (args.Length == 0)
        {
            Console.WriteLine("Usage: {0}.exe <command>", Assembly.GetExecutingAssembly().GetName().Name);
            return;
        }

        string @params = "";
        if (args.Length > 1)
            @params = string.Join(" ", args, 1, args.Length - 1);

        ProcessStartInfo psi = new ProcessStartInfo(args[0], @params);
        psi.Verb = "runas";
        Process.Start(psi);
    }
}

This tool uses the regular Process class from the .NET Framework but specifies a Verb on the ProcessStartInfo object. You can find more info on verbs in the Windows SDK under "Verbs and File Associations". Enjoy!

Del.icio.us | Digg It | Technorati | Blinklist | Furl | reddit | DotNetKicks

You've probably already heard about this new feature in .NET 3.5: extension methods. If not, check out a few posts on this topic:

As a quick recap, extension methods allow you to "extend" an existing type with additional methods, without touching the type's implementation itself. In fact, it's nothing more than syntactical sugar on top of static methods. The code below shows such a set of extension methods written in C# 3.0:

using System;

namespace MyExtensions
{
    public static class Extensions
    {
        private static Random rand = new Random();

        public static string Reverse(this string s)
        {
            char[] c = s.ToCharArray();
            Array.Reverse(c);
            return new string(c);
        }

        public static string Permute(this string s, int n)
        {
            char[] c = s.ToCharArray();

            for (int i = 0; i < n; i++)
            {
                int a = rand.Next(s.Length);
                int b = rand.Next(s.Length);
                char t = c[a];
                c[a] = c[b];
                c[b] = t;
            }

            return new string(c);
        }

        public static int Power(this int i, int n)
        {
            int res = 1;
            for (int j = 0; j < n; j++)
                res *= i;
            return res;
        }
    }
}

These methods allow you to write the following, if the MyExtensions namespace is imported:

string name = "Bart";
string reverseName = name.Reverse(); //traB
string fuzzyName = name.Permute(1); //can produce different results where two letters are switched, e.g. Brat
int n = 2;
int kb = n.Power(10); //1024

So far so good. But what does all of this magic have to do with PowerShell? The answer: nothing (so far). As you know, Windows PowerShell 1.0 was developed in the .NET 2.0 timeframe, so it would be very strange if PowerShell understood extension methods. And indeed, it doesn't ... which made me think how we could make this available using some plumbing. Here's what I came up with.

Windows PowerShell has a feature called the Extended Type System. This allows types to be extended with additional properties, methods, etc in order to provide additional IT admin convenience. For example, take a look at the types.ps1xml file in the %windir%\system32\WindowsPowerShell\v1.0 folder on your system. In there you'll find things like:

<Type>
    <Name>System.Array</Name>
    <Members>
        <AliasProperty>
            <Name>Count</Name>
            <ReferencedMemberName>Length</ReferencedMemberName>
        </AliasProperty>
    </Members>
</Type>

This defines an "alias property" which adds a property called Count to each instance of System.Array, pointing to the Length property available on System.Array. So, you can write this:

> $a = "Bart", "John"
> $a.Count
2

In fact, if you use get-member on $a, you'll see the AliasProperty listed out there (click to enlarge):

image

In a similar way, one can make different types of extensions: Alias Properties, Code Properties, Note Properties, Script Properties, Code Methods, Script Methods. Take a closer look at types.ps1xml for additional samples. Back to our mission now. It seems ETS is an appropriate vehicle to make extension methods available using Script Methods. Basically, we'll provide a script for each extension method that takes the set of original parameters and rewrites these to become parameters of the static method. For example, if you write:

> $name = "Bart"
> $name.Reverse()

the last call should become:

> [MyExtensions.Extensions]::Reverse($name)

Similarly, a Power call on an int should be translated from:

> $n = 2
> $n.Power(10)

into:

> [MyExtensions.Extensions]::Power($n, 10)

Taking possible method overloads into account, we should end up with something like this:

<?xml version="1.0" encoding="utf-16"?>
<Types>
  <Type>
    <Name>System.String</Name>
    <Members>
      <ScriptMethod>
        <Name>Reverse</Name>
        <Script>
          switch ($args.Count) {
            0 { [MyExtensions.Extensions]::Reverse($this) }
            default { throw "No overload for Reverse takes the specified number of parameters." }
          }
        </Script>
      </ScriptMethod>
      <ScriptMethod>
        <Name>Permute</Name>
        <Script>
          switch ($args.Count) {
            1 { [MyExtensions.Extensions]::Permute($this, $args[0]) }
            default { throw "No overload for Permute takes the specified number of parameters." }
          }
        </Script>
      </ScriptMethod>
    </Members>
  </Type>
  <Type>
    <Name>System.Int32</Name>
    <Members>
      <ScriptMethod>
        <Name>Power</Name>
        <Script>
          switch ($args.Count) {
            1 { [MyExtensions.Extensions]::Power($this, $args[0]) }
            default { throw "No overload for Power takes the specified number of parameters." }
          }
        </Script>
      </ScriptMethod>
    </Members>
  </Type>
</Types>

Once we have such a file, it can be "imported" in Windows PowerShell using the Update-TypeData cmdlet:

image

All we have to do is call this cmdlet as follows:

> Update-TypeData -prependPath MyExtensions.ps1xml

But it gets even better: this chunk of XML is something that's an ideal candidate for dynamic code XML generation. Guess what, let's use LINQ for this task and wrap the "extension method export" functionality in a custom cmdlet:

using System;
using System.IO;
using System.Linq;
using System.Management.Automation;
using System.Reflection;
using System.Runtime.CompilerServices;
using System.Text;
using System.Xml;
using System.Xml.Linq;

namespace BdsSoft.PowerShell.ExtensionMethods
{
    [Cmdlet("Import", "ExtensionMethods")]
    public class ImportExtensionMethods : PSCmdlet
    {
        [Parameter(Mandatory=true)]
        public Assembly Assembly { get; set; }

        [Alias("ns"), Parameter(Mandatory=false)]
        public string Namespace { get; set; }

        protected override void ProcessRecord()
        {
            if (Namespace == null)
                Namespace = "";

            var res =
                new XDocument(
                    new XElement("Types",
                        from t in Assembly.GetTypes()
                        where t.Namespace != null && t.Namespace.StartsWith(Namespace)
                        from m in t.GetMethods(BindingFlags.Public | BindingFlags.Static)
                        where m.GetCustomAttributes(typeof(ExtensionAttribute), false).Length == 1
                        group m by m.GetParameters()[0].ParameterType into g
                        select
                            new XElement("Type",
                                new XElement("Name", g.Key.FullName),
                                new XElement("Members",
                                    from m in g
                                    group m by m.Name into h
                                    select
                                        new XElement("ScriptMethod",
                                           new XElement("Name", h.Key),
                                            new XElement("Script", GetScriptFor(h))
                                    )
                                )
                            )
                    )
                );

            StringBuilder sb = new StringBuilder();

            using (TextWriter tw = new StringWriter(sb))
            {
                using (XmlTextWriter xtw = new XmlTextWriter(tw))
                {
                    xtw.Indentation = INDENT;
                    xtw.Formatting = Formatting.Indented;
                    res.WriteTo(xtw);
                }
            }

            base.WriteObject(sb.ToString());
        }

        static int INDENT = 2;

        static string GetScriptFor(IGrouping<string, MethodInfo> m)
        {
            StringBuilder sb = new StringBuilder();
            sb.AppendFormat("\r\n{0}switch ($args.Count) {{\r\n", new String(' ', INDENT * 5));

            foreach (var e in m)
            {
                int n = e.GetParameters().Length - 1;
                sb.AppendFormat("{0}{1} {{ {2} }}\r\n", new String(' ', INDENT * 6), n, GetScriptFor(e.DeclaringType.FullName, e.Name, n));
            }

            sb.AppendFormat("{0}default {{ throw \"No overload for {1} takes the specified number of parameters.\" }}\r\n", new String(' ', INDENT * 6), m.Key);
            sb.AppendFormat("{0}}}\r\n{1}", new String(' ', INDENT * 5), new String(' ', INDENT * 4));

            return sb.ToString();
        }

        static string GetScriptFor(string type, string method, int n)
        {
            StringBuilder sb = new StringBuilder();
            sb.Append("$this");

            for (int i = 0; i < n; i++)
                sb.AppendFormat(", $args[{0}]", i);

            string args = sb.ToString();

            return String.Format("[{0}]::{1}({2})", type, method, args);
        }
    }
}

Quite a bit of code, but lots of plumbing to get a smooth output (notice the code can be improved in many places). The GetScriptFor methods are pretty simple to understand and generate the script for a given (group of) method (overloads) associated with a method name (the second GetScriptFor method is a helper to get the method calls themselves, using the $this and $args variables). For what the core functionality is concerned, take a look at the ProcessRecord method that contains a LINQ query that looks for all extension methods in the given assembly and namespace:

            var res =
                new XDocument(
                    new XElement("Types",
                        from t in Assembly.GetTypes()
                        where t.Namespace IsNot Nothing AndAlso t.Namespace.StartsWith(Namespace)
                        from m in t.GetMethods(BindingFlags.Public | BindingFlags.Static)
                        where m.GetCustomAttributes(typeof(ExtensionAttribute), false).Length == 1
                        group m by m.GetParameters()[0].ParameterType into g
                        where !g.Key.IsGenericType
                        select
                            new XElement("Type",
                                new XElement("Name", g.Key.FullName),
                                new XElement("Members",
                                    from m in g
                                    group m by m.Name into h
                                    select
                                        new XElement("ScriptMethod",
                                           new XElement("Name", h.Key),
                                            new XElement("Script", GetScriptFor(h))
                                    )
                                )
                            )
                    )
                );

In here, we're using the new System.Xml.Linq API to contruct an XML fragment on the fly (LINQ to XML style). A method is considered to be an extension method if it's public, static and marked with a System.Runtime.CompilerServices.ExtensionAttribute custom attribute. Furthermore, it's first parameter (the "this" parameter in C# 3.0) shouldn't be a generic type, since PowerShell doesn't have first-level support for generics at the moment (this restriction rules out the use of System.Core's extension methods unfortunately, at the moment). Using the grouping constructs, methods are grouped per type and per method name (to account for overloads). VB folks have an easier job when it comes down to generating XML fragments. The query above looks as follows in VB 9.0 (more info on VB 9.0 XML integration):

image

When writing a cmdlet, we need a snap-in as its distribution vehicle; below is a simple one:

using System.ComponentModel;
using System.Management.Automation;

namespace BdsSoft.PowerShell.ExtensionMethods
{
    [RunInstaller(true)]
    public class ExtensionMethodsSnapIn : PSSnapIn
    {
        public override string Description
        {
            get { return "This Windows PowerShell snap-in provides support for .NET Framework 3.5 Extension Methods."; }
        }

        public override string Name
        {
            get { return "BdsSoft.PowerShell.ExtensionMethods"; }
        }

        public override string Vendor
        {
            get { return "BdsSoft"; }
        }
    }
}

Strong-name the assembly, build it and run installutil -i against the generated dll file. Finally, create the following PowerShell script, ImportExtensionMethods.ps1:

if ($args.Count -lt 2)
{
   throw "Usage: .\ImportExtensionMethods.ps1 output assembly [namespace]"
}
else
{
   $emOutput = (join-path (split-path $profile) $args[0])

   Import-ExtensionMethods -assembly $args[1] -namespace $args[2] | Out-File $emOutput
   Update-TypeData -PrependPath $emOutput
}

This simplifies calling the Import-ExtensionMethods cmdlet and to update the ETS type data, so that the extensions become available. In the screenshot below, you can see the whole thing in use:

image

All you need to do is:

  1. Make sure the snap-in is loaded, using add-pssnapin (you can move this call to your PowerShell profile if you want to load it automatically).
  2. Call ImportExtensionMethods.ps1, passing in a name for the ETS ps1xml file that should be generated followed by the assembly that contains the extension methods (use Assembly.LoadFrom or Assembly.Load to get the assembly either by file name or by assembly name, i.e. for GAC'ed assemblies).

Of course, once you have the ps1xml files, you could simply adapt your PowerShell profile file in order to load the assemblies and call Update-TypeData to load the ps1xml file. In the next screenshot you can see the generated ps1xml file (click to enlarge):

image 

Have fun!

Del.icio.us | Digg It | Technorati | Blinklist | Furl | reddit | DotNetKicks

Welcome back to the Visual Basic 9.0 Feature Focus blog series. In this post, we'll (finally) cover one of .NET Framework 3.5's core features for both C# 3.0 and VB 9.0: LINQ. LINQ stands for Language INtegrated Query and provides querying syntax directly in a general-purpose programming language. Combined with clean and elegant APIs like LINQ to Objects, LINQ to SQL and LINQ to XML (amongst others, like LINQ to SharePoint) this gives developers all the horsepower they need to retrieve and manipulate data from any LINQ-capable data source, also closing the gap between various data models and objects from the world of OO (for instance, by having entity mappings for relational SQL databases, a.k.a. O/R mapping).

So, what's the big idea? In the past, there were various problems related to working with data. One of those problems is the programming language's inawareness of the concept of queries. For example, you typically write code like this:

Using conn As New SqlConnection(dsn)
    Using cmd As New SqlCommand("SELECT * FROM Products WHERE UnitPrice > 100 ORDERBY UnitsInStock DESCENDING", conn)
        Using reader = cmd.ExecuteReader()
            While reader.Read()
                Dim name As String = CType(reader("ProductName"), String)
                Dim price As Decimal = CType(reader("UnitPrice"), Decimal)
                'Use it
            End While
        End Using
    End Using
End Using

What's wrong with this? A couple of things in fact. First of all, the SQL statement is wrapped inside a string which makes it impossible for the compiler to analyze it and check it against the database schema for correctness (well, in theory we could have additional compiler intelligence, like the printf format string checking in many C/C++ compilers, but that doesn't solve the fundamental problem). Secondly, inside the iteration statement, we're faced with weak typing problems: ProductName was a string, is masked as an object and has to be casted back to a String in order to work with it (which doesn't seem too much of a problem if you're working with late binding semantics, but it would be better if we could have strong typing bubbling up to the surface). Another typical problem is query statement parameterization which can be done but (in lots of case) wrong as well. What I'm referring at is a phenomenon known as SQL Injection Attacks that occurs if you're building query statements by simple string concatenation, instead of using parameterized statements or sprocs. Last but not least there's all of this awful plumbing around the query statement in order to execute it and to iterate over the results, which by the way aren't real results: it's a bit of loosely coupled data that you still have to tie together using some O/R mapping to live in an ideal world. So, how can this be improved?

The approach taken by LINQ is unique in many fashions, the core asset being the LIN part: Language INtegrated. Furthermore, with LINQ we do have one single syntax for all sorts of queries, including in-memory ones and queries targeting various other kinds of data sources, like SQL, XML, web services like Amazon, SharePoint, ... In this post, we'll take a closer look at the query syntax and what it corresponds to in terms of extension method calls. Be prepared to get a big deja-vu of most features we did discuss in previous posts, including Extension Methods, Lambda Expressions, Expression Trees, Anonymous Types, Object Initializers, etc. Let's start with a very simple query using LINQ to Objects, otherwise known as in-memory queries:

Dim ints As Integer() = {1, 2, 3, 4, 5}
Dim res = From i In ints Where i Mod 2 = 0 Select i
For Each i In res
    Console.WriteLine(i)
Next

Note: A common question I often get is why the query starts with a From clause, in contrast to SQL statements. The reason why isn't difficult at all: by having the From clause first, the IDE knows about the source you're about to query. Since the source should be an IEnumerable(Of T), each time you refer to an object from that source using the dummy name from the From clause (the .. part in From .. In ...), it knows that one is of type T (from IEnumerable(Of T)). So, the editor can provide IntelliSense on the query statements. If you were to start with a Select clause, how can the editor know what the members are of i (Select i. --> can you suggest any meaningful IntelliSense, ignoring the System.Object members of course?).

The query will return all even numbers, but how does it really work? Let's do what the compiler does: translate the query to another format:

Dim res = ints.Where(Function(i) i Mod 2 = 0).Select(Function(i) i)

Essentially this is a chain of extension method calls on the IEnumerable(Of T) object, combined with lambda expressions. In compiling the original query, the compiler looks for the dummy variable in the From .. In clause and uses that one as the parameter to all subsequent lambdas. Next it takes the expressions followed by Where .., Select .., etc and cooks a lambda out of it, by taking the dummy as the parameter and the original expression as the lambda body. Finally it chains the whole thing together using extension methods calls. Now on to these extensions: what does Where and Select do? First of all, both are extensions to IEnumerable(Of T) and have been defined in System.Linq.Enumerable. So, the code above is equal to:

Dim res = Enumerable.Select(Enumerable.Where(ints, Function(i) i Mod 2 = 0), Function(i) i)

Note: you might wonder why there's a Select call with a trivial projection lambda Function(i) i. If the original item is returned, what's the value of this? The answer is to make a query really a query, i.e. to 'defeat' degenerate queries. If a trivial query like "From i In ints Select i" would be optimized to just "ints" (because the projection doesn't do anything), the source of the query is revealed to the consumer of the query, something which has to be avoided in order to have a clean design (and which is more important when talking about LINQ to SQL kind of queries too).

Quite funny isn't it, to see the order of the methods calls being reversed due to the use of extension methods (tip: people who saw a decent amount of lambda calculus and formal semantics can write down some nice formulas to generalize this idea). The signature of these extension methods is the following:

Function Where(Of T)(ByVal source As IEnumerable(Of T), ByVal predicate As Func(Of T, Boolean)) As IEnumerable(Of T)
Function Select(Of T,R)(ByVal source As IEnumerable(Of T), ByVal projection As Func(Of T, R)) As IEnumerable(Of R)

Both generic methods take in a source as well as a function delegate that transforms the input to something else: the predicate returns a Boolean (should the item be included or not?) and the projection returns an object of another type R (the result of projection T into something else, e.g. an anonymous type).

Evaluation starts with the Enumerable.Where call, which takes the source from its first parameter and iterates over it, evaluating the lambda expression "predicate" for each and every item in the source sequence. If that predicate evaluates to true, it returns the item in the result sequence, which also is of type IEnumerable(Of T). Unfortunately I can't give you sample code of how the Where method works since Visual Basic doesn't have a concept called "iterators" which lies at the basis of this mechanism and is a C# 2.0 (and higher) feature. What is enables is a thing called "lazy evaluation": only when you start iterating over the results of the method, data is fetched from the source. It's like having a pipeline that sucks data out of the source only when it's really needed:

image

 

In the picture above imaging you start iterating over the results of the Select (the end of the query in the sample). As you turn the Select wheel, the Where weel will start to spin around, getting data from the source sequence till a match is found. Then that match bubbles up all the way to the Select output. So, in the source sequence, the following will happen:

1, 2, 3, 4, 5

You start to iterate over Select, which comes down to calling MoveNext on the enumerator (the thing that drives the For Each loop). Which enumerator? The one returned by the Select call. Internally, the Select method sucks data from the Where method by calling the Where method's returned enumerator's MoveNext method. This on its turn triggers an iteration over the source sequence. We've just arrived at the bottom of the call stack, now let's make our way up again:

  • 1 is fetched from the source
    • Where evaluates the predicate and sees 1 doesn't match
  • 2 is fetched from the source
    • Where evaluates the predicate and sees 2 is a match, so it yields back the 2 to its caller
      • Select got 2 from its source, it executes the lambda on it and returns the result (the lambda "projection" being Function(i) i, thus returning i itself)
        • The caller of the iteration receives 2

It's until the caller asks for another iteration cylce that the whole thing is suspended. That's why we call this lazy loading. Assume you have a source sequence of 1 million items and you break the iteration after five cycles: you won't have loaded any more data from the source sequence than is required. This is different than doing simple collection manipulations, reading the entire source sequence and finding all elements that are even, then taking this result and doing the projection for each of those items. In such a case you'd end up with a maximum of 1 million items (all the odd numbers are filtered out), maybe 500,000 or so, but you just needed 5 in the final iteration. So you've just spilled lots of processor cycles and memory space to do more than you really needed to

Note: There are situations however where lazy loading is broken because all elements are needed to carry out some operations. Notable examples are sorting and grouping. Indeed, to sort a sequence of items that have been ordered at random, all items have to be fetched in order to carry out a sort, before the result sequence can be returned to the caller.

So far so good for in-memory queries. What about other queries, like LINQ to SQL queries? Again, the mechanism is very similar. However, a query like the following:

Dim ctx As New NorthwindDataContext()
Dim res = From p In ctx.Products Where p.UnitPrice > 100 Select p.ProductName
For Each p In res
    Console.WriteLine(p)
Next

The first stage of the compilation is the same as previously:

Dim res = ctx.Products.Where(Function(p) p.UnitPrice > 100).Select(Function(p) p.ProductName)

Now it becomes interesting. As you know, Where and Select are extension methods but how does the compiler find the appropriate one? The answer is simple: by looking at the variable on which the method is called. In this case ctx.Products is of type Table(Of T), a class defined in the System.Data.Linq namespace. Table(Of T) doesn't have an instance method called Where, so the compiler starts to look for extension methods, a search based on the type of the object. There are no extension methods for Table(Of T) in scope but since Table(Of T) implements IQueryable(Of T), a class from the System.Linq namespace, the compiler looks for extension methods on IQueryable(Of T) and finds that one is available now (hooray). The signature of this method (and the Select method, since Queryable.Where returns an IQueryable(Of T) again) is:

Function Where(Of T)(ByVal source As IQueryable(Of T), ByVal predicate As Expression(Of Func(Of T, Boolean))) As IQueryable(Of T)
Function Select(Of T,R)(ByVal source As IQueryable(Of T), ByVal projection As Expression(Of Func(Of T, R))) As IQueryable(Of R)

So, the query is now translated into:

Dim res = Queryable.Select(Queryable.Where(ctx.Products, Function(p) p.UnitPrice > 100), Function(p) p.ProductName)

If you read the method signatures carefully, you'll see that the type of res is now IQueryable(Of String), because p.ProductName (the result value of the method call chain produced by the original query) is a string. However, there's more. Notice the second parameters of both methods: these are no longer of type Func(Of ...) but of type Expression(Of Func(Of ...)), which - as we learned before in the episode on Expression Trees - trigger the compiler to generate expression trees for the lambdas. So,

Function(p) p.UnitPrice > 100
Function(p) p.ProductName

both become expression trees, passed in to the IQueryable(Of T) object. Now, when you trigger iteration over the query (results), like this:

For Each p In res
    Console.WriteLine(p)
Next

or indirectly, e.g. by doing databinding sort of operations, you'll cause the IQueryable(Of T) implementation to parse the expression trees and transform these into the appropriate target language, in this case SQL:

SELECT p.ProductName FROM Products WHERE p.UnitPrice > 100

Now you know the basic principles, it's time to take a look at the various query operators that are supported by LINQ (yes, there's more than just Wheres and Selects):

From indicates the source of a query and has the following syntax:

From <dummy> [As <type>] In <source>

    • dummy is a variable name for use throughout the query to denote an item being processes (e.g. in a predicate or in a projection);
    • type can be used to indicate the type of the dummy, i.e. the type of an item in the source sequence; it's not mandatory if the type of the item can be inferred (e.g. when using generic collections IEnumerable(Of T) or IQueryable(Of T));
    • source obviously references the query source, e.g. an in-memory collection or an IQueryable(Of T) object.

Note: More than one From clause can be used in a query.

Select defines a projection clause in a query:

Select <projection>

    • projection can be any expression that returns one object; it could be a dummy from a From clause or something more complex, using the syntax below:

[<alias> = ] <column>[,...]

    • alias is optional and indicates the name of the property that will be generated on the anonymous type representing the projection;
    • column refers to a property or field on a dummy but could be a more complex expression too (in which case an alias is required since no name can be inferred automatically).

Examples:

  • Dim res = From i In New Integer() {1, 2, 3, 4, 5, 6} Select i 'returns 1,2,3,4,5,6
  • Dim res = From i In New Integer() {1, 2} Select Double = 2 * i, Triple = 3 * i 'returns {Double = 2, Triple = 3}, {Double = 4, Triple = 6}

Where is used to filter results based on a predicate:

Where <predicate>

    • predicate is a Boolean-valued expression that can use every variable in scope, including the dummies defined in From clauses.

Example: Dim res = From i In New Integer() {1, 2, 3, 4, 5, 6} Where i Mod 2 = 0 Select 'returns 2,4,6

Order By clauses put an ordering on the query's results:

Order By <ordering> [Ascending | Descending]

    • ordering is an expression used a the key in an ordering; if multiple orderings are specified in a row, subsequent orderings define secondary, ternary, ... and n-ary orderings (for the curious ones, primary orderings are translated into OrderBy and OrderByDescending calls, while n-ary orderings are translated into ThenBy and ThenByDescending calls).

Example:

  • Consider src contains { ID = 1, City = "Ghent", Country = "Belgium" }, { ID = 2, City = "Brussels", Country = "Belgium" }, { ID = 3, City = "Seattle", Country = "USA" }
  • Dim res = From c In src Order By c.Country Descending Order By c.City Select c.ID 'returns 3,2,1

Distinct filters out duplicates from a source sequence in a query:

Distinct

    • returns all distinct items from the input of the clause (i.e. "everything in front of" the Distinct clause)

Example: Dim res = From i In New Integer() {1, 2, 1, 3} Select Distinct 'returns 1,2,3

Skip and Take allow to select a range of items from a source sequence in a query:

Take(<number>)
Skip(<number>)

    • number is an integer value; e.g. Skip(10).Take(5) will select items 11 to 15 from the input sequence

Example: Dim res = From i In New Integer() {1, 2, 3, 4, 5, 6} Select Skip Take 'returns 4,5

Aggregate performs an aggregate, i.e. turning a sequence of items into a singleton value:

Aggregate <dummy> In <source> Into <aggregationList>

    • dummy is a dummy name for use in the aggregation;
    • source can be anything which is enumerable (including a nested subquery);
    • aggregationList is a comma-separated set of aggregation expressions that take the form:

      [<alias> =] <aggregation>[,...]

      If more than one aggregation expression is present, an anonymous type will be generated using the aliases used in the aggregation expression list. The aggregation supports various functions including Min, Max, Sum, Average, ...

Example: Dim res = Aggregate i In (From i In New Integer() {1, 2, 3, 4, 5, 6} Where i Mod 2 = 0 Select i) Into a = Average(), s = Sum() 'returns res.a = 4, res.s = 12

Group is used to make groupings based on a grouping expression; this is slightly more complicated than the query operations above:

Group [<fields>] By <keys> Into <aggregates>

    • fields is an optional comma-separated list and allows to specify the fields that have to be included in the grouped results (you could consider this to be a pre-grouping Select-alike projection);
    • keys is a comma-separated list of one or more name = value pairs, specifying the grouping keys;
    • aggregates is a comma-separated list of expressions that make up the group result and allow for continuation of the query.

A sample will clarify things so much:

Dim res = From s In New String() {"Bart", "Bob", "John"} Group By FirstLetter = s(0) Into Words = Group
For Each letter In res
    Console.WriteLine(letter.FirstLetter)
    For Each word In letter.Words
        Console.WriteLine("- " + word)
    Next
Next

Notice we don't have a fields part in here, but in most cases queries as the one above are complex enough for day-to-day purposes. In this sample, a set of words are grouped by the first letter (FirstLetter) of the word and the items in the group are called Words. Inside the loop, we can refer to FirstLetter as the grouping key and Words as the collection of the items in the corresponding group.

Join allows to join multiple sources together based on some join condition:

Join <dummy> In <source> On <key1> Equals <key2> [And ...]

    • dummy refers to items in the to-be-joined source;
    • source is the source of the items that have to be joined;
    • key1 and key2 are expressions to used in the join condition.

An example based on Northwind looks like this:

Dim res = From p In ctx.Products Join s In ctx.Suppliers On p.Supplier = s Select p.ProductName, s.SupplierName

Joins can be more complex if using "group joins" or when multiple join conditions are involved but let's keep it simple for now :-).

Happy coding!

Del.icio.us | Digg It | Technorati | Blinklist | Furl | reddit | DotNetKicks

Welcome back to the Visual Basic 9.0 Feature Focus blog series. In this post, we'll cover a feature unique to Visual Basic: XML Support. In VB 9.0, XML has become a first class citizen of the language, meaning you can use XML literals to build XML fragments (sounds pretty logical isn't it?) together with a few operators that make working with XML really simple (and I mean really simple). Combine this with support for XSD schemas and you get a superb language feature. Let's take a closer look...

First of all, take a look at the Inline XML syntax:

Dim xml = <Books>
                      <Book Title="Introducing VB 9.0" Price="0.00">
                          <Author>Bart De Smet</Author>
                      </Book>
                      <Book Title="Introducing C# 3.0" Price="0.00">
                          <Author>Bart Simpson</Author>
                      </Book>
                  </Books>

Not very exciting is it? Just plain XML inside the program, combined with and Implicitly Type Local Variable. So, what's the type of the xml variable in the fragment above? Drumroll ... it's an XElement. No, not an XmlElement, but an XElement without the 'ml'. XElement is defined in a new .NET Framework 3.5 namespace, called System.Xml.Linq that's included by default in new VB 9.0 applications. So, why do we have a new XML API in .NET? The answer is simple: make things easier and composable. If there's one thing the System.Xml implementation - based on the DOM - lacks, it's composability. Imagine you have to construct the XML piece from the code above using the System.Xml API. You'd end up with spaghetti like this:

Dim doc As New XmlDocument()
Dim books As XmlElement = doc.CreateElement("Books")

Dim book1 As XmlElement = doc.CreateElement("Book")
Dim title1 As XmlAttribute = doc.CreateAttribute("Title")
title1.Value = "Introducing VB 9.0"
book1.Attributes.Append(title1)
Dim title2 As XmlAttribute = doc.CreateAttribute("Price")
title2.Value = "0.00"
book1.Attributes.Append(title2)
Dim author1 As XmlElement = doc.CreateElement("Author")
author1.InnerText = "Bart Simpson"
book1.AppendChild(author1)
books.AppendChild(book1)

The code fragment above just defines one single book by the way... I do have only one question for you: can you still smell the XML in the code above? Apart from the language integration in VB 9.0, the new System.Xml.Linq API is much more composable. To illustrate this, consider the fragment below:

Dim xml = New XElement("Books", _
            New XElement("Book", _
                New XAttribute("Title", "Introducing VB 9.0"), _
                New XAttribute("Price", "0.00"), _
                New XElement("Author", "Bart De Smet")), _
            New XElement("Book", _
                New XAttribute("Title", "Introducing C# 3.0"), _

                New XAttribute("Price", "0.00"), _
                New XElement("Author", "Bart Simpson")))

This piece of code defines the same as the original fragment that uses VB 9.0's Inline XML feature. I'm sure you agree this, compared to the System.Xml code, exposes the XML structure quite fairly well. In fact, the code shown above just has one constructor call at the top level, constructing the "Books" element and specifying the child items. Even more, guess what a call to ToString() returns? Yes, indeed:

image

That's about it for the System.Xml.Linq API right now, feel free to play around with it and let IntelliSense be your guide. You'll find it very easy to create new pieces of XML even using the raw APIs (that is, without Inline XML in case you want to do the work yourself) and to get to the data. But how can we make this even easier by means of language integration? Let's explore. What about this:

Dim xml = <Books>
                      <Book Title="Introducing VB 9.0" Price="0.00">
                          <Author>Bart De Smet</Author>
                      </Book>
                      <Book Title="Introducing C# 3.0" Price="0.00">
                          <Author>Bart Simpson</Author>
                      </Book>
                  </Books>

For Each book In xml.<Book>
    Console.WriteLine("{0} costs {1} and is written by {2}", book.@Title, book.@Price, book.<Author>.Value)
Next

Whoosh, can it be any easier than this? Just iterate over every <Book> in the XML fragment and select some values. That's right: you can use the dot '.' syntax (the child axis) to dive in the XML structure (a <Book> has an <Author>, thus you can write book.<Author>) and the '.@' syntax (the attribute axis) to select attributes. So, the code above prints:

Introducing VB 9.0 costs 0.00 and is written by Bart De Smet
Introducing C# 3.0 costs 0.00 and is written by Bart Simpson

However, there's something that can be improved in the code above: writing it. Take a look at the IntelliSense:

image

Although a few syntax elements are outlined, there's not much help from the IDE at this stage of the game. That's because the IDE doesn't have information about the XML schema, so how to solve this? The answer, again, is pretty simple: import an XML schema, just like you import a CLR namespace. But before we can do se, we need to create the schema first. An easy way to do this, is by copying the XML fragment straight from the code to an XML file in Visual Studio. Then go to XML, Create Schema:

image

Next, copy the generated schema to the Clipboard, add new item of type XML Schema to the solution, named "VB9Demo.xsd":

image

Copy in the XSD schema that's still on the clipboard and make sure to add the targetNamespace attribute:

image

Save the schema file and go back to the code file and add the following import on top of the code:

Imports <xmlns="http://tempuri.org/VB9Demo">

As you'll see while typing, IntelliSense pops up and helps you a bit:

image

 Now, when you start to use the xml variable, IntelliSense will be so kind to help you a bit:

image 

and

image 

Notice it's also possible to denote an XML namespace by using the following:

Imports <xmlns:ns="http://tempuri.org/VB9Demo">

See the :ns part in the Imports statement above? Now you can use <Books>, <Book> and <Author> if you prefix these with ns:, just as in plain XML. If you don't use a prefix, the schema will be considered to be the default one. A final note on IntelliSense for XML: in VB 9.0 this is limited to XML variables that have been constructed already and that have a known namespace in place. For example, when writing a piece of Inline XML, you won't get IntelliSense. This is something the team is looking into for a subsequent release but I think all of this magic is already very very powerful (and you might want to type the XML pieces in a regular XML editor - like Visual Studio itself - that's aware of the schema you're using, so you get IntelliSense when writing the XML; finally you can copy the result right into your code).

There's one operator I didn't mention yet, that's the ... operator (the descendant axis):

For Each author In xml...<Author>
    Console.WriteLine(author.Value)
Next

This will look for all <Author> tags no matter how deep these are nested in the XML structure, which is certainly easier than having to traverse the XML tree manually or using the '.' syntax.

If you think all of this is already very cool on its own, wait for the next post in this series where we'll cover LINQ, including the combination with XML features in VB 9.0 (a.k.a. the return of the <% ... %>).

Happy coding!

Del.icio.us | Digg It | Technorati | Blinklist | Furl | reddit | DotNetKicks

Welcome back to the Visual Basic 9.0 Feature Focus blog series. In this post, we'll cover Runtime Agility. Before we continue, I have to warn you: this is a HUGE feature even though it hasn't been covered that much yet in the blogosphere. Historically, Visual Basic has been a language that's highly dependent on a runtime library. People from the old VB days will certainly remember msvbvmxx.dll, the Microsoft Visual Basic Virtual Machine version xx. Not only did this library contain runtime functionality (e.g. for memory management), it also did contain helper functions that are used to give operations in Visual Basic the right semantics (e.g. how a string compare works in VB). As we moved into the .NET era, the need popped up to keep the semantics consistent compared to the past. That's why Visual Basic .NET requires a runtime library called Microsoft.VisualBasic.dll which can be found in the GAC. So far so good, but what about the implications of this? Right, every VB.NET application has a dependency on this runtime library, which implies the library has to be redistributed and loaded and maybe there's lots of stuff inside Microsoft.VisualBasic.dll that your app is never going to use. Sol, what about eliminating this dependency? Enter runtime agility!

Let's start to outline the current situation with a simple sample. Compile the following using the VB 8.0 compiler (either through the IDE or using vbc.exe:

Class Program
   Shared Sub Main()
   End Sub
End Class

Now, when you take a look at the generated IL code metadata using ILDASM.exe you'll see the following:

image

Even for an application as easy (that's an understatement) as the above, the Microsoft.VisualBasic assembly is referenced by the compiler. Now in Visual Basic 9.0, things are changing, also because of the growing need for a lightweight runtime (e.g. in the context of Silverlight). Now it's possible to compile applications without the Visual Basic runtime or with a different runtime. Using the Visual Basic 9.0 compiler, compile the code above with the following command:

vbc -vbruntime- demo.vb

This will produce the following metadata:

image

As you can see, there's no dependency on the Microsoft.VisualBasic assembly anymore. But what if you start to use functionality that requires the runtime library, like this:

Class Program
   Shared Sub Main(ByVal args As String())
      Console.WriteLine(args(0) = args(1))
   End Sub
End Class

In here, we're performing a string comparison, which gets translated into a CompareString method call, a function that's defined inside the Microsoft.VisualBasic assembly:

image

So, if you try to compile this without the VB runtime, you'd end up with the following compile error message:

image

The cool thing is you can build your own runtime library that just has the functionality you need. For example, I'll be using C# to write a small VB Runtime library that just has a CompareString operator:

using System;

namespace Microsoft.VisualBasic.CompilerServices
{
   public class Operators
   {
      public static int CompareString(string left, string right, bool textCompare)
      {
         return 0;
      }
   }
}

The code above is for demo purposes only: it will say the strings are equal no matter what the input is (0 = equal, negative = left < right, positive = left > right). Compile this library using:

csc -t:library vbruntime.cs

Alternative you could write the runtime implementation in VB itself too:

Namespace Microsoft.VisualBasic.CompilerServices
   Public Class Operators
      Public Shared Function CompareString(ByVal left As String, ByVal right As String, ByVal textCompare As Boolean) As Integer
         Return 0
      End Function
   End Class
End Namespace

which can be compiled using:

vbc -vbruntime- -t:library vbruntime.vb

Finally, compile the Visual Basic demo code using:

vbc -vbruntime:vbruntime.dll demo.vb

Now, everything should work fine and you should see the following in the metadata of the generated assembly:

image

And in the Main function, our CompareString operation is used:

image

When you execute the program, you'll get True now matter what the two input arguments are (see our custom 'bad' implementation):

>demo Bart John
True

Tip: Try to compile an average-sized application using the -vbruntime- flag to see how much runtime dependencies there are.

Happy coding!

Del.icio.us | Digg It | Technorati | Blinklist | Furl | reddit | DotNetKicks

Welcome back to the Visual Basic 9.0 Feature Focus blog series. In this post, we'll cover Relaxed delegates, a small but useful feature that boosts developer convenience. In VB 8.0 the use of delegates was very strict concerning the methods supported to be called through a delegate. Two samples make this clear:

Delegate Sub Process(ByRef o As Object)

Module Module1

    Sub DoIt(ByRef s As String)
        Console.WriteLine(s)
    End Sub

    Sub Main()
        Dim p As New Process(AddressOf DoIt)
        p("Hello")
    End Sub

End Module

Oops, this doesn't work. But why? The IDE and compiler complain about the following:

image

What's up? VB 8.0 has the nasty habit to require an exact signature match when pointing a delegate to a method. In the sample above, the compiler complains that the parameter to DoIt is a String while it expects an Object, even though each String is an Object because of the class hierarchy. In other words, regular rules that apply to regular method calls do not hold anymore when working with delegates: signatures have to match exactly. The same holds for the use of event handlers:

Class FrontEnd

    Private WithEvents engine As MyEngine

    Private Sub OnError(ByVal source As Object, ByVal e As ErrorEventArgs) Handles engine.OnError

    End Sub

End Class

Class MyEngine

    Public Event OnError As EventHandler(Of ErrorEventArgs)

    Public Sub Fail()
        RaiseEvent OnError(Me, New ErrorEventArgs("Bang!"))
    End Sub

End Class

Class ErrorEventArgs
    Inherits EventArgs

    Private _message As String

    Public ReadOnly Property Message() As String
        Get

            Return _message
        End Get
    End Property

    Public Sub New(ByVal message As String)
        _message = message
    End Sub

End Class

The code above will compile but if you change the OnError method signature of the FrontEnd class like this:

    Private Sub OnError(ByVal source As Object, ByVal e As EventArgs) Handles engine.OnError

it won't. See the difference? The 'e' parameter now is of type EventArgs rather than ErrorEventArgs and although the class hierarchy is there to make this a valid signature in regular method call circumstances, the compiler will nag about it:

image

The solution? VB 9.0! Both fragments above will compile now in VB 9.0 thanks to the concept of relaxed delegates. In other words, the method signature use in a delegate binding shouldn't be an exact match anymore, it should just be consistent with the rules that apply for regular method invocations. But there's more... If you don't need the arguments of the method being bound to the delegate, either using AddressOf or Handles, you can omit these:

Sub DoIt(ByRef s As String)

will be accepted for the following delegate:

Delegate Sub Process(ByRef o As Object)

and:

Private Sub OnError() Handles engine.OnError

will be accepted for this delegate:

EventHandler(Of ErrorEventArgs)
= Delegate Sub(ByVal sender As Object, ByVal e As ErrorEventArgs)

Happy coding!

Del.icio.us | Digg It | Technorati | Blinklist | Furl | reddit | DotNetKicks

Welcome back to the Visual Basic 9.0 Feature Focus blog series. In this post, we'll cover Friend Assemblies, something that has been supported by C# (and the .NET Framework more generally) since Whidbey (see here). What this feature allows you to do is to expose Friend types (which are by definition visible inside the assembly in which they've been defined) to another assembly that's considered to be a "friend" of the defining assembly.

This is useful in situations where you want to split up functionality across multiple assemblies that still require tight interaction. While the outside world can only see Public types, assemblies with a friendship relation can see each other's Friend types as well. Another practical situation where this proves to be pretty useful is when writing debugger visualizers that ship with some library: you might want to make the debugger visualizer a friend of the library so that it can use internal 'Friend' features or access internal 'Friend' data that needs to be visualized, while keeping a nice separation between the library itself and the debugger visualizer (to reduce memory footprint for example).

So, how does it work? Assume you've created a Class Library project called "MyLibrary" that contains the following:

Friend Class Bar

End Class

Next, you have a Console project called "Demo" that has a reference to MyLibrary. It's pretty trivial to see that the following won't work:

Imports MyLibrary

Module Demo

    Sub Main()
        Dim b As New Bar()
    End Sub

End Module

Indeed, the IDE tells us:

image

Now go back to the MyLibrary project and change the code as follows:

Imports System.Runtime.CompilerServices

<Assembly: InternalsVisibleTo("Demo")>

Friend Class Bar

End Class

Now, the code in the Demo project will compile fine since MyLibrary was so kind to expose its internals to us. The InternalsVisibleToAttribute (of which you can have more than one) takes the assembly name as its first parameter, which can be supplied in various formats: just the name (without the .dll or .exe extension) or the full name (name, version, public key token, culture).

That's it for now; just make sure you have (damn) good reasons to use this feature: don't spill friendship relations.

Happy coding!

Del.icio.us | Digg It | Technorati | Blinklist | Furl | reddit | DotNetKicks

More Posts