Feeds:
Posts
Comments

Archive for the ‘MongoDB’ Category

Fluent MongoDB Part 3

In Parts 1 and 2 of this series we built a simple fluent interface that allowed us to save, fetch, and delete domain objects in a MongoDB data store.

At this point you’re probably wondering whether you really have to build out all the fluent tooling yourself just to be able easily store and fetch your domain objects. Well, it depends on the API you want to use. You could roll it yourself as we’ve been doing or you could go with an existing library. It turns out mongodb-csharp does come with a fluent query interface called MongoDB.Linq that provides the ability to build queries from expressions. As we’ve seen, however, mongodb-csharp does not currently provide strongly typed collections so we’d still end up going with a hybrid implementation requiring lots of non-business support code.

Alternatively, we could try a different C# adapter for MongoDB – one that provides both strongly typed collections that can use our domain objects, and the ability to query the store using Linq. An adapter that meets those requirements is NoRM and in this Part we’ll be rebuilding our test app using that tool. Let’s go.

Once again we’ll start with our Person domain object.

[Serializable]
public class Person
{
	[MongoIgnoreIfNull]
	public ObjectId Id { get; set; }
	[MongoIgnoreIfNull]
	public string FirstName { get; set; }
	[MongoIgnoreIfNull]
	public string LastName { get; set; }
}

You’ll notice a couple of additions to the version we had in Part 1. The first is the Id property and second is the MongoIgnoreIfNull attribute on all the nullable properties. If you’ve gone much beyond where we were in Part 2 you’ll have realized that you really need a property to hold the MongoDB-id for the object in order to perform efficient queries. We have the same constraint in NoRM.

The other problem you may have encountered while trying mongodb-csharp is explicitly querying on a null property. The NoRM solution to that problem is to decorate the appropriate properties with a hint as shown above.

Now to drive out some functionality. To start with, we’ll build an instance of the Person:

	var person = new Person
	{
		FirstName = "Loreena",
		LastName = "McKennitt"
	};

Next we’ll get get some hooks for our database and Person data store.

	_server = new Mongo("tests");
	_people = _server.GetCollection<Person>();

NoRM takes care of translating our object to something MongoDB understands, so all we have to do is call Save():

	_people.Save(person);

Last we need to verify that the object ends up with a MongoDB-id:

	person.Id.ShouldNotBeNull();

Super simple. Next let’s drive out the query implementation. Once again NoRM provides the FirstOrDefault() funcionality so we just have to get a queryable collection and call it. If you want to use Linq in NoRM you have to get an instance of MongoQuery. Note: This means we have to use two different objects to interact with the database, one for saving and another for queries.

MongoQuery wants a MongoQueryProvider to query against and that object wants the name of the database, not the name of the collection. This has implications on table naming so it might be best not to give an explicit name when getting a typed collection.

For now we’ll keep it simple and create whichever type we need on the fly:

	[SetUp]
	public void BeforeEachTest()
	{
		_provider = new MongoQueryProvider("tests");
		_server = _provider.Server;
	}

	[Test]
	public void Should_be_able_to_fetch_by_example()
	{
		var person = new Person
		{
			FirstName = "Loreena",
			LastName = "McKennitt"
		};

		_server.GetCollection<Person>().Save(person);
		person.Id.ShouldNotBeNull();

		person = new MongoQuery<Person>(_provider)
			.FirstOrDefault(x => x.FirstName == "Loreena");
		person.ShouldNotBeNull();
		person.LastName.ShouldBeEqualTo("McKennitt");
	}
}

The MongoIgnoreIfNull attribute comes into play when we call FirstOrDefault or any other Linq query method. If we hadn’t used that attribute the query would have been translated to something equivalent to:

select top 1
from Person
where FirstName = "Loreena"
and LastName is null
and Id is null

Not what I wanted at all… but now we have a different problem. Since the inclusion of null in the query is determined by an attribute, we can’t change it on the fly. Perhaps I really did want the person who didn’t have a last name. Just something to think about.

Let’s move on. NoRM provides two methods for deleting. There’s Delete(T object) that wants an object that already exists in the data store – it is going to use the Id property.

	_server.GetCollection<Person>().Delete(person);

And there’s Delete<T>(T object) that wants a template for the kinds of things you want to delete.

	_server.GetCollection<Person>()
		.Delete<Person>(new Person
					{
						FirstName = "Loreena"
					});

This is place where you might accidentally call the wrong one and delete nothing or a lot more than expected. I’d be inclined to put an extension like DeleteByExample around the template version and maybe DeleteById on the other to make it clear what is going to happen.

Also note that we’re using the Collection again, so it is used for both Save() and Delete().

The NoRM interface provides all the capabilities we wanted out-of-the-box but can intrude a bit into our domain objects in order to support advanced queries. There’s a lot more to play with in this library so do give it a whirl.

Download everything you need to get started including all the code in this post.

Advertisements

Read Full Post »

Fluent MongoDB Part 2

In Part 1 of this series we built a simple fluent interface that allowed us to save domain objects to MongoDB. In this part we’ll tackle fetching and deletion.

As we did before, we’ll start with a test to drive out the functionality we’re looking for:

	[TestFixture]
	public class When_fetching
	{
		private (something) _people;

		[Test]
		public void Should_be_able_to_fetch_by_example()
		{
			var person = new Person
			{
				FirstName = "Loreena",
				LastName = "McKennitt"
			};

			string id = _people.Save(person);
			id.ShouldNotBeNull();

			person = _people.FirstOrDefault(x => x.FirstName == "Loreena");
			person.ShouldNotBeNull();
			person.LastName.ShouldBeEqualTo("McKennitt");
		}
	}

The problem here is, if we implement FirstOrDefault() as an extension method on IMongoCollection:

	public static class IMongoCollectionExtensions
	{
		public static T FirstOrDefault<T>(
				this IMongoCollection collection, 
				Expression<Func<T, bool>> func) where T : class
		{
			...;
		}

we’ll have to specify the type in the call:

	var person = _people.FirstOrDefault<Person>(x => x.FirstName == "Loreena");

If we want it to work like Linq then the implementation of FirstOrDefault() should already know that x is a Person. This means _people should also be strongly typed. It is clear that IMongoCollection is not going to work because that interface isn’t generic so we’ll build a strongly typed wrapper – here’s the desired usage:

	public void BeforeEachTest()
	{
		var mongo = new Mongo();
		mongo.Connect();
		var db = mongo.getDB("tests");
		_people = db.GetCollection<Person>("people");
	}

This implies an extension method:

	public static class DatabaseExtensions
	{
		public static ITypedCollection<T> GetCollection<T>(this Database database,
		                                                   string collectionName)
			where T : class
		{
			return new TypedCollection<T>(database.GetCollection(collectionName));
		}
	}

with ITypedCollection stubbed as:

	public interface ITypedCollection<T>
	{
		T FirstOrDefault(Expression<Func<T, bool>> func);
	}

The func has to be an Expression because we’re going to want not only the value of the Property but also its name. This implies some static reflection so at this point I’m going to include a reference to mvba-core so I can make use of its Reflection class.

And to continue the call chain, we’ll store the IMongoCollection in the TypedCollection constructor:

	public TypedCollection(IMongoCollection collection)
	{
		_collection = collection;
	}

Now let’s think about FirstOrDefault(). We’ll have to get the Property name and value somehow into a Document so we can call _collection.FindOne() to find by example. If it returns null so do we. Otherwise we received a Document and have to convert it to the expected type.

	public T FirstOrDefault(Expression<Func<T, bool>> func)
	{
		var spec = GetSpecification(func);
		var result = _collection.FindOne(spec);

		return result == null ? null : result.ConvertTo<T>();
	}

For building the specification we’re going to really simplify the implementation by only supporting PropertyName == value comparisons. We’ll let mvba-core’s Reflection class do the work of figuring out the property name and value for us.

	private static Document GetSpecification(Expression<Func<T, bool>> func)
	{
		var binaryExpression = func.Body as BinaryExpression;
		if (binaryExpression == null)
		{
			throw new ArgumentException("Only supports: (x=>x.Property == value) at the moment");
		}
		var comparisonType = binaryExpression.NodeType;
		if (comparisonType != ExpressionType.Equal)
		{
			throw new ArgumentException("don't know how to build specification for comparison NodeType: " + comparisonType);
		}

		var propertyName = Reflection.GetPropertyName(binaryExpression.Left as MemberExpression);
		var value = Reflection.GetValueAsString(binaryExpression.Right);

		return new Document { { propertyName, value } };
	}

Finally, we need to convert the Document we received to the requested type. This looks like another place where we can be lazy and let Json.NET do all the work.

	public static class DocumentExtensions
	{
		public static T ConvertTo<T>(this Document document)
			where T : class
		{
			if (document == null)
			{
				return null;
			}

			string json = JsonConvert.SerializeObject(document, Formatting.None);
			var result = JsonConvert.DeserializeObject<T>(json);
			return result;
		}
	}

That’s it. Oops, we have a problem. We haven’t implemented Save() on TypedCollection yet so our fetch test won’t compile. Let’s just do that real quick. First update the Save() tests to reflect the new paradigm:

	public void BeforeEachTest()
	{
		var mongo = new Mongo();
		mongo.Connect();
		var db = mongo.getDB("tests");
		_people = db.GetCollection<Person>("people");
	}

Add Save to ITypedCollection:

	public interface ITypedCollection<T>
	{
		T FirstOrDefault(Expression<Func<T, bool>> func);
		string Save(T item1);
	}

And move the Part 1 Save() implementation from an extension method to TypedCollection then delete IMongoCollectionExtensions:

	public string Save(T item)
	{
		var dictionary = item.ToDictionary();
		var document = dictionary.ToDocument();
		_collection.Update(document);
		return document["_id"].ToString();
	}

Both our Save tests and fetch test should pass.

Now just one more integration test for FirstOrDefault() to make sure it can handle the case where null is returned by the Database.

	public void Should_get_null_if_there_is_no_match()
	{
		var person = _people.FirstOrDefault(x => x.LastName == "Marley");
		person.ShouldBeNull();
	}

So we’ve implemented a very specific case of FirstOrDefault(condition) that would probably meet 80 percent of our needs in a simple environment. Let’s move on and implement Delete().

We’ll create a domain object and store it. Then fetch it back to verify that it exists. Then delete it with a lambda and finally verify that it no longer exists. You see now why we needed to implement FirstOrDefault() first… it was necessary for verifying that Delete() works.

	public void Should_be_able_to_delete_an_object_by_example()
	{
		var person = new Person
		{
			FirstName = "Loreena",
			LastName = "McKennitt"
		};

		_people.Save(person);

		person = _people.FirstOrDefault(x => x.FirstName == "Loreena");
		person.ShouldNotBeNull();
		person.LastName.ShouldBeEqualTo("McKennitt");

		_people.Delete(x => x.FirstName == "Loreena");
		person = _people.FirstOrDefault(x => x.FirstName == "Loreena");
		person.ShouldBeNull();
	}

Here we’ll take advantage of the infrastructure we built for FirstOrDefault() and add Delete() to ITypeCollection as well:

	public interface ITypedCollection<T>
	{
		void Delete(Expression<Func<T, bool>> func);
		T FirstOrDefault(Expression<Func<T, bool>> func);
		string Save(T item1);
	}

Then implement as follows:

	public void Delete(Expression<Func<T, bool>> func)
	{
		var spec = GetSpecification(func);
		_collection.Delete(spec);
	}

All done.

Download everything you need to get started including all the code in this post.

This infrastructure works for simple domain objects and queries but it won’t be long before you’ll start to feel its limitations and want to do more complex Linq queries. Go ahead and experiment then check out Part 3 when you are ready for more.

Read Full Post »

Fluent MongoDB Part 1

If you’ve played with MongoDB you’ve discovered that communicating with MongoDb via Documents is painful. It is painful because we don’t care about Documents. We care about storing our domain objects and having powerful ways of fetching them again from the data store. How do we accomplish those tasks when then tool doesn’t offer those options?

The answer is, of course, we extend the tool so that we can express our intent in the most natural language we can. To me this means Linq or some reasonable subset thereof.

So lets start with the domain object Person:

	[Serializable]
	public class Person
	{
		public string FirstName { get; set; }
		public string LastName { get; set; }
	}

The object is marked Serializable but is otherwise unremarkable. If I have an instance of person I’d like to be able to simply pass it to a Save method on the collection as follows.

	var person = new Person
	{
		FirstName = "Loreena",
		LastName = "McKennitt"
	};
	_people.Save(person);

We can’t change the IMongoCollection object directly but we can use an extension method to build the interface we’re looking for:

	public static class IMongoCollectionExtensions
	{
		public static void Save<T>(this IMongoCollection collection, T item) where T: class
		{
			// magic happens
		}
	}

Now to drive out the magic part. Fundamentally the data in the object we want to save has to be translated to something MongoDB understands – a Document. We’ll take this in stages by translating first to a Dictionary then to a Document. Here are some tests to drive out the functionality of a ToDictionary() extension.

	public class TExtensionsTests
	{
		[TestFixture]
		public class When_asked_to_convert_a_class_to_a_Dictionary
		{
			private const string FirstNameKey = "FirstName";
			private const string FirstNameValue = "Loreena";
			private const string LastNameKey = "LastName";
			private const string LastNameValue = "McKennitt";
			private Person _person;

			[SetUp]
			public void BeforeEachTest()
			{
				_person = new Person
				{
					FirstName = FirstNameValue,
					LastName = LastNameValue
				};
			}

			[Test]
			public void Should_associate_property_values_with_their_names()
			{
				var result = _person.ToDictionary();

				result[FirstNameKey].ShouldBeEqualTo(FirstNameValue);
				result[LastNameKey].ShouldBeEqualTo(LastNameValue);
			}

			[Test]
			public void Should_store_property_names_as_keys()
			{
				var result = _person.ToDictionary();

				result.ContainsKey(FirstNameKey).ShouldBeTrue();
				result.ContainsKey(LastNameKey).ShouldBeTrue();
			}
		}
	}

We could implement this with some reflection by recursively iterating over the Properties and Fields in the Person and transferring their values to a Dictionary but for now we’ll be lazy and take advantage of another tool to do the heavy lifting for us for now – Json.NET. Go ahead and download a copy to your library and add a reference to your project.

	public static class TExtensions
	{
		public static IDictionary<string,object> ToDictionary<T>(this T item) where T: class
		{
			string json = JsonConvert.SerializeObject(item, Formatting.None);
			var dictionary = JsonConvert.DeserializeObject<Dictionary<string, object>>(json);
			return dictionary;
		}
	}

Next we’ll move the Dictionary contents into the Document. Since Document doesn’t expose anything useful that can consume a Dictionary or IDictionary, we have to add the key-value pairs one-by-one. First a convenience extension for performing an action on all items in an IEnumerable:

	public static class IEnumerableExtensions
	{
		public static IEnumerable<T> DoToEach<T>(this IEnumerable<T> items, Action<T> action)
		{
			foreach(var item in items)
			{
				action(item);
			}
			return items;
		}
	}

Next more tests to drive out the implementation of the ToDocument() extension:

	public class IEnumerableKeyValuePairOfStringToObjectExtensionsTests
	{
		[TestFixture]
		public class When_asked_to_convert_to_a_Document
		{
			private const string FirstNameKey = "FirstName";
			private const string FirstNameValue = "Loreena";
			private const string LastNameKey = "LastName";
			private const string LastNameValue = "McKennitt";
			private Dictionary<string, object> _dictionary;

			[SetUp]
			public void BeforeEachTest()
			{
				_dictionary = new Dictionary<string, object>
				{
					{ FirstNameKey, FirstNameValue }, 
					{ LastNameKey, LastNameValue }
				};
			}

			[Test]
			public void Should_associate_key_values_with_their_names()
			{
				var result = _dictionary.ToDocument();

				result[FirstNameKey].ShouldBeEqualTo(FirstNameValue);
				result[LastNameKey].ShouldBeEqualTo(LastNameValue);
			}

			[Test]
			public void Should_copy_all_keys_to_the_Document()
			{
				var result = _dictionary.ToDictionary();

				result.ContainsKey(FirstNameKey).ShouldBeTrue();
				result.ContainsKey(LastNameKey).ShouldBeTrue();
			}
		}
	}

Then the implementation:

	public static class IEnumerableKeyValuePairOfStringToObjectExtensions
	{
		public static Document ToDocument<T>(this T items) where T : IEnumerable<KeyValuePair<string,object>>
		{
			var document = new Document();
			items.DoToEach(x => document.Add(x.Key, x.Value));
			return document;
		}
	}

And now the magic part of the Save method:

	public static class IMongoCollectionExtensions
	{
		public static string Save<T>(this IMongoCollection collection, T item) where T : class
		{
			var dictionary = item.ToDictionary();
			var document = dictionary.ToDocument();
			collection.Update(document);
			return document["_id"].ToString();
		}
	}

Now if we call the Save method it will be persisted in the collection but how will we know it worked? Well, we should write some tests around the Save() method to verify that _people.Update() is called and populated correctly. I’ll leave that as an exercise for the reader for now. One thing we can do is write an integration test to check the return value of the Save() method.

	[TestFixture]
	public class When_trying_basic_tasks_fluently
	{
		private IMongoCollection _people;

		[SetUp]
		public void BeforeEachTest()
		{
			var mongo = new Mongo();
			mongo.Connect();
			var db = mongo.getDB("tests");
			_people = db.GetCollection("people");
		}

		[Test]
		public void Should_be_able_to_store_an_object()
		{
			var person = new Person
			{
				FirstName = "Loreena",
				LastName = "McKennitt"
			};
			var dbId = _people.Save(person);
			dbId.ShouldNotBeNull();
		}
	}

That’s enough for now. In Part 2 of this post we’ll delve into how to reverse this process to get a Person back from the collection instead of a Document.

Read Full Post »

MongoDB C# Quickstart

So you’ve heard of object storage databases and would like to get started playing with one in a familiar language – C#. Me too, let’s go:

First create following directory as this is where MongoDB expects to store data by default:

c:\data\db

Next download the latest build of MongoDB. The MongoDB executables are all self contained and no install is required. The only required exe is bin/mongod.exe – the server side of the MongoDB equation. If this exe isn’t running then your code won’t work. You could copy it to your desktop, or make a quicklaunch link, but I suggest copying it to c:\data\db and making a quicklaunch link from there so that everything is colocated for now.

The first time you start MongoDB your firewall may warn you that it has blocked this program from accepting incoming network connections. You must unblock it or you won’t be able to use MongoDB.

If you like MongoDB then later on you will want to run it as a service so you don’t have to think about starting the exe. To do that call it with the –install parameter.

Great now that MongoDB is running how do we talk to it in C#? For that we need a library. A popular library for that purpose is mongodb-csharp. Download and extract the mongodb-csharp files to your common library (wherever you usually put such things) as we’ll need some of them as references in our test project.

I tend to use NUnit to try out new tools because it helps me to quickly build up a library of sample code that demonstrates capabilities of the tool for later reference, so that’s how I’ll begin here too.

Create a new library project in Visual Studio.

Add a reference to NUnit’s nunit-framework.dll from your common library.

Add a reference to mongodb-csharp\MongoDB.Driver.dll

Next I add a reference to FluentAssert so that I can abstract away some of the NUnit semantics.

Now to start coding.

First we need to connect to MongoDB:

	var mongo = new Mongo();
	mongo.Connect();

We’ll tell it we want to use the object store named “tests”. This “database” doesn’t exist but MongoDB will just create it on the fly as we use it so don’t worry about that part.

	var db = mongo.getDB("tests");

Let’s start with the Person concept as People have lots of useful attributes that require different datatypes like Name, Age, DateOfBirth, etc. We”ll store those in a “People” collection. Again, this does not exist in our object store at first but MongoDB takes care of that detail for us.

	_people = db.GetCollection("people");

Now remember, we’re fundamentally working with object storage. MongoDB calls these Documents and mongodb-csharp maintains that verbiage. Think of Documents as string-to-object mapping containers for now.

Lets do a bare metal Save test with a Document. First create a Document and add some data to it.

	var person = new Document();
	person["FirstName"] = "Loreena";
	person["LastName"] = "McKennitt";

Then add it to the People object collection.

	_people.Insert(person);

How do we know it worked? Documents have an _id property that uniquely identifies them in the object store. If the Insert was successful we should see a value in this property after the call to Insert() .

	person["_id"].ShouldNotBeNull();

Here’s the entire test:

	public class SaveTests
	{
		[TestFixture]
		public class When_trying_basic_tasks
		{
			private IMongoCollection _people;

			[SetUp]
			public void BeforeEachTest()
			{
				var mongo = new Mongo();
				mongo.Connect();
				var db = mongo.getDB("tests");
				_people = db.GetCollection("people");
			}

			[Test]
			public void Should_be_able_to_store_an_object()
			{
				var person = new Document();
				person["_id"].ShouldBeNull();

				person["FirstName"] = "Loreena";
				person["LastName"] = "McKennitt";

				_people.Insert(person);
				person["_id"].ShouldNotBeNull();
			}
		}
	}

Go ahead and experiment with storing other types in your Document.

	person["BirthDate"] = new DateTime(1957, 2, 17);

Now how do we fetch something from the object store? One way is to give MongoDB an example of what you want. Let’s ask for a Person named Loreena.

	[Test]
	public void Should_be_able_to_fetch_by_example()
	{
		var spec = new Document();
		spec["FirstName"] = "Loreena";

		var person = _people.FindOne(spec);
		person.ShouldNotBeNull();
		person["LastName"].ShouldBeEqualTo("McKennitt");
	}

Huh. That appears to have worked but is it just faking us out by returning the only thing on the shelf? Let’s verify by asking for something that we know is not in the store.

	[Test]
	public void Should_get_null_if_there_is_no_match()
	{
		var spec = new Document();
		spec["LastName"] = "Marley";

		var person = _people.FindOne(spec);
		person.ShouldBeNull();
	}

Next we’ll explore object deletion. Objects can also be deleted from the object store by example. We’ll check that the Person named Loreena exists (due to our previous test) then delete her and verify that she was deleted by trying to fetch her again.

	[Test]
	public void Should_be_able_to_delete_an_object_by_example()
	{
		var spec = new Document();
		spec["FirstName"] = "Loreena";

		var person = _people.FindOne(spec);
		person.ShouldNotBeNull();
		person["LastName"].ShouldBeEqualTo("McKennitt");

		_people.Delete(spec);
		person = _people.FindOne(spec);
		person.ShouldBeNull();
	}

That’s the gist of it. There’s quite a bit more power available in the by-example concept that we haven’t explored yet but this should be enough to get you started.

Download everything you need to get started including all the code in this post.

Now when you get to the point where Documents start to get in the way because you really want to store your objects come back and check out my posts on working with MongoDB fluently.

Read Full Post »

%d bloggers like this: