Archive for the ‘.NET’ Category
DDPClient.NET–.NET client for Meteor’s Distributed Data Protocol
Last couple of hours I was working on a small .NET client library to connect to Meteor js application using Distributed Data Protocol (DDP). This post will give some insight to the library I am working on. You can see more details of DDP here.
Using DDPClient.NET you can subscribe to published items or call a Meteor method and display the same in your ASP.NET or Desktop applications.
First let’s go through the server code in my meteor application.
if(Meteor.is_server) { Meteor.publish("allproducts", function(){ return Products.find(); });
Meteor.startup(function(){
console.log("starting Point of sale application.....");
});
Meteor.methods({
addProduct: function (prodCode, prodDesc) {
return "Product Name: " + prodDesc + " Product Code: " + prodCode;
}
});
}
From the Meteor application we are publishing an item called ‘allproducts’ and have a server method called ‘addProduct’ that takes two parameters. Now let’s see how to subscribe to published item and call the server method using DDPClient.NET.
Subscribe to Published items
class Program { static void Main(string[] args) { IDataSubscriber subscriber = new Subscriber(); DDPClient client = new DDPClient(subscriber); client.Connect("localhost:3000"); client.Subscribe("allproducts"); } }
public class Subscriber:IDataSubscriber { public void DataReceived(dynamic data) { try { if (data.type == "sub") { Console.WriteLine(data.prodCode + ": " + data.prodName +
": collection: " + data.collection); } } catch(Exception ex) { throw; } } }
As you can see it’s very easy to connect to Meteor application using DDPClient.NET. Just call the Connection function with the url. Then call the subscribe function to subscribe to any published item in Meteor application.
In the above code I subscribed to ‘allproducts’ item. After you subscribed successfully to the meteor application, we will receive all the products stored in the db. Once our client is subscribed any insert/delete/update will streamed to the .NET client. You can test it by adding a some products via the web application and you can see the newly added product get displayed in the console.
How to call a Meteor Method?
class Program { static void Main(string[] args) { IDataSubscriber subscriber = new Subscriber(); DDPClient client = new DDPClient(subscriber); client.Connect("localhost:3000"); client.Call("addProduct", "NS5", "IRobot"); Console.ReadLine(); } }
public class Subscriber:IDataSubscriber { public void DataReceived(dynamic data) { try { if (data.type == "method") Console.WriteLine(data.result); } catch(Exception ex) { throw; } } }
As you can see the Meteor method ‘addProduct’ take two arguments prodCode and prodDesc. So when we call the method we have to pass the parameter as well. We can do that by invoking the ‘Call’ function with the method name and arguments as shown below.
client.Call(“addProduct”, “NS5”, “IRobot”);
Also you will get notification if any data get deleted from the database. You can check the same as shown below.
else if (data.type == "unset") { Console.WriteLine("deleleted item with id: " + data.id);
}
so you will get the id of the data it deleted, you can search for the product with this id and remove or do what ever you wanted to do.
DDPClient.NET is checked into Github. The meteor code I referred here is added to the one I used in Getting Started with meteor.
Transparent Persistence with db4o
I was working on a small Invoicing application for my brother and I uses db4o for data persistence. Again I used db4o here because of it’s simplicity and ease of doing future updates.
I started with POCO entities and db4o handles the insertions and deletion quiet well. I started experiencing trouble when I try to update object graphs. For e.g. I created a SaleOrder with two Items then I retrieve the order and added one more item and save the changes. db4o will never save the newly added Item, it always shows the items that added initially. Because db4o couldn’t make out the changes in the object graph. There are two ways to let db4o handle this scenario.
UpdateDepth
There is a problem with UpdateDepth, we have to define a depth say 5, 10, etc and db4o will traverse the graph to identify the changes, that means we should forecast our object graph depth. This also introduce performance issues. So I decided not to use it.
Transparent Persistence
With Transparent Persistence (TP) the object will tell which all the fields got modified and db4o will update the changed fields efficiently.
To enable Transparent Persistence we have to do some changes in
Let’s start with repository, I used the same kind of repository I created for iTraveller with some changes in creating the db4o context/container. The change is as shown below.
public IObjectContainer Context { get { if (_context == null) this.CreateContext(); return _context; } } public void CreateContext() { if (_context == null) { IEmbeddedConfiguration config = Db4oEmbedded.NewConfiguration(); config.Common.Add(new TransparentActivationSupport()); config.Common.Add(new TransparentPersistenceSupport()); _context = Db4oEmbedded.OpenFile(config, DBNAME); } }
It’s the time to do some modification in Entity level.
using System;
usingDb4objects.Db4o.Activation;
usingDb4objects.Db4o.Collections;
usingDb4objects.Db4o.TA;
namespaceNet.Sales.Manager.Domain
{
public classSaleOrderHeader : IActivatable
{
[System.NonSerialized]
privateIActivator _activator;
publicSaleOrderHeader ()
{
this.OrderDate = DateTime.Now;
this.SaleOrderItems = newArrayList4<SaleOrderItem>();
}
public int OrderId { get; set; }
private IList<SaleOrderItem> _salesOrderItems;
public IList<SaleOrderItem> SaleOrderItems
{
get
{
this.Read();
return this._salesOrderItems;
}
private set
{
this._salesOrderItems = value;
this.Write();
}
}
private double _discount;
public double Discount
{
get
{
this.Read();
return _discount;
}
set
{
this.Write();
this._discount = value;
}
}
public void With(SaleOrderItem saleOrderItem)
{
if(this.SaleOrderItems==null)
this.SaleOrderItems = new ArrayList4<SaleOrderItem>();
this.Read();
this.SaleOrderItems.Add(saleOrderItem);
}
public void Activate(ActivationPurpose purpose)
{
if (this._activator != null)
_activator.Activate(purpose);
}
public void Bind(IActivator activator)
{
if (_activator == activator)
return;
if (activator != null && null != _activator)
throw new System.InvalidOperationException();
_activator = activator;
}
public void Read()
{
this.Activate(ActivationPurpose.Read);
}
public void Write()
{
this.Activate(ActivationPurpose.Write);
}
}
}
For simplicity I removed some functions and properties from the above class. As you can see the SalesOrderHeader is implemented IActivatable. All classes that implements IActivatable is a candidate for Transparent Activation and Transparent Persistence. IActivatable will add two methods Activate and Bind. Make sure you implement the functions as shown above. I also added two helper methods Read and Write to ease the use of Activate function. Have a look at the property Discount and see how we are using it.
private double _discount; public double Discount { get { this.Read(); return _discount; } set { this.Write(); this._discount = value; } }
It is as simple as that.
If we a have collection and that needs TA and TP enabled then we have to use ArrayList4. It’s inherited from IList. In the above e.g. SaleOrderItem is a collection, see how we implemented it.
private IList<SaleOrderItem> _salesOrderItems; public IList<SaleOrderItem> SaleOrderItems { get { this.Read(); return this._salesOrderItems; } private set { this._salesOrderItems = value; this.Write(); } }
Now we have to implement IActivatable in SaleOrderItem entity as well, the approach is same as mentioned above so skipping the code.
We are done, now db4o can handle Transparent Activation and Transparent Persistence and no need to provide any graph depth. Even if we provide Update or Activate depth db4o will ignore it.
Note
We don’t need to implement IActivatable in top level class if it’s not dealing with a collection. Say for e.g. we have Customer and Address class and Customer has an instance of Address. Then implement IActivatable in Address no need to implement it in Customer as by default db4o will update top level object.
Query TA/TP enabled objects
You can see that in my Generic Repository I query data using Lambda expression, and the query is created as shown below.
public IList<TEntity> Find<TEntity>(Func<TEntity, bool> criteria) where TEntity : class { return this.Query<TEntity>().Where<TEntity>(criteria).ToList<TEntity>(); }
private IList<TEntity> Query<TEntity>() where TEntity : class { return this.Context.Query<TEntity>(); }
_repository.Find<SaleOrderHeader>(soh => soh.OrderId == salesOrderNumber);
This Query model is not working if we enable TA and TP. First or Find of SaleOrder always return null. I think I need to check with db4o devs.
The alternative is use Query by Example (QBE) as shown below.
public IList<TEntity> Find<TEntity>(TEntity criteria) where TEntity : class { return this.Context.QueryByExample(criteria).Cast<TEntity>().ToList<TEntity>(); }
public SaleOrderHeader getSalesOrder(int salesOrderNumber) { return _repository.First<SaleOrderHeader>(new SaleOrderHeader(salesOrderNumber)); }
You can find more details from db4o online help.
I will be releasing the entire source to Github soon.
NHibernate Error – ‘nhIdOutParam’ No size set for variable length data type
I am working on a project where I have to connect to an Oracle database. I used NHibernate as my ORM, every thing worked as expected until the Save operation. The table I was dealing with has a ROWID field an auto generated field by Oracle. Here we cannot use the native generator of NHibernate, I felt the best fit here is ‘trigger-identity’ generator. As shown below I configured the rowid in my hbm file.
<id name="_rowId" column="RowId" access="field"> <generator class="trigger-identity"></generator> </id>
To my surprise Nunit throws the error “System.Exception : Parameter ‘:nhIdOutParam’: No size set for variable length data type: String.” whenever I try to save the entity.
I analyzed the insert query generated by NHibernate and every things is fine. Next option for me is google, like every one else I started searching in google, I got so many forums where people reported this issue but no solid answer. The error stack trace shows that the error is from System.Data.OracleClient, I felt like the issue is with the Oracle client I am using, here I am using the client from Microsoft not the native ODP.Net provided by Oracle. It seems like system.data.OracleClient is unable to handle the ‘returning’ statement added by NHibernate.
Nhibernate will create the Insert statement as shown below
Insert into Table (filed1, field2) values (p1, p2) returning RowId into :nhIdOutParam
I changed the NHibernate configuration to use the Oracle client and referred Oracle.DataAccess.dll to my project. Compiled the project and run the NUnit test, wow I got the Green bar after 4hrs of struggling. In my previous NHibernate configuration (hibernate.cfg.xml) the driver class property was
<property name="connection.driver_class">NHibernate.Driver.OracleClientDriver</property>
This configuration forces NHibernate to use the Oracle client provided with .NET. To instructed NHibernate to use the native oracle assembly (Oracle.DataAccess.dll) change the configuration as below
<property name="connection.driver_class">NHibernate.Driver.OracleDataClientDriver</property>
Hope this post may help some one to save some time.
Persist Static typed Entities to Neo4j using Neo4jD
At the core Neo4jD uses Node and Relationship to persist data to Neo4j graph database. If you want to see how the core is working then visit here. You might notice that, the Node and Relationship is not static typed objects. Most of us work with Static typed entities and want to persist the same. To deal with static typed entities I added a layer on top of Neo4jD core called NodeMapper.
Node Mapper
Below are the features of NodeMapper
- Map the entity to Node object and persist in Neo4j.
- Handles the relation between entities, if an entity has a sub entity then NodeMapper will create a relationship between them.
- Persist the object graph using the Model created, will cover later in the page.
- Inject interceptor for Lazy loading of related entities.
- And more
How to Persist entities using NodeMapper
To persist the entities, first we need to draw the object graph of the Entity we need to persist. Let’s have a look at an eg.
public class Order { public Order() { _orderItems = new List<OrderItem>(); } [EntityId] public int Id { get; set; } public virtual string Name { get; set; } IList<OrderItem> _orderItems; public virtual IList<OrderItem> OrderItems { get { return _orderItems; } private set { _orderItems = value; } } public void AddOrderItem(OrderItem item) { this._orderItems.Add(item); } } public class OrderItem { public OrderItem() { } public OrderItem(int id, Product product) { this.Id = id; this.Product = product; } [EntityId] public int Id { get; set; } public virtual Product Product { get; set; } } public class Product { public Product() { } public Product(int id, string productName) { this.Id = id; this.ProductName = productName; } [EntityId] public int Id { get; set; } public string ProductName { get; set; } }
You will notice some rules in defining entities.
-
The unique Id field is decorated with EntityId attribute. It’s mandatory to identify the Id field of the entity.
-
Properties that needs Lazy loading should be virtual.
-
Mandatory to have a default parameter less constructor.
Now let’s define the object graph. This configuration helps NodeMapper to persist the entire Graph. It’s very similar to how we do in Entity Framwork Code first approach.
public class OrderConfiguration:EntityConfiguration<Order> { public OrderConfiguration() { this.RelatedTo<OrderItem>(o => o.OrderItems); } } public class OrderItemConfiguration:EntityConfiguration<OrderItem> { public OrderItemConfiguration() { this.RelatedTo<Product>(oi => oi.Product); } } public class ProductConfiguration:EntityConfiguration<Product> { public ProductConfiguration() { this.Leaf(); } }
As you can see in the ProductConfiguration, there is no related entity so we marked it as Leaf.
Let’s see how to persist an order
[SetUp] public void Initialize() { GraphEnvironment.SetBaseUri("http://localhost:7474/"); ModelBuilder.Add(new OrderConfiguration()); ModelBuilder.Add(new OrderItemConfiguration()); ModelBuilder.Add(new ProductConfiguration()); }
[TestCase] public void SaveOrder() { Order order = new Order(); order.Id = 0; order.Name = "Sony"; order.AddOrderItem(new OrderItem(0, new Product(0, "Rice"))); order.AddOrderItem(new OrderItem(0, new Product(0, "Sugar"))); NodeMapper nodeMapper = new NodeMapper(); nodeMapper.Save<Order>(order); Console.WriteLine(order.Id.ToString()); Assert.AreEqual(1, order.Id); }
In the Initialize method of NUnit Test we added EntityConfigurations to ModelBuilder. NodeMapper uses the ModelBuilder to understand the Object graph and uses reflection to traverse through the object graph and persist it.
Lazy Loading
Neo4jD uses Lazy loading to load the related entities, it uses Castle DynamicProxy to intercept property calls and inject lazy loading functionality. To perform lazy loading the property should be virtual, you can see the OrderItems in Order entity.
Retrieve saved Entity
[TestCase] public void GetOrder() { NodeMapper nodeMapper = new NodeMapper(); Order order = nodeMapper.Get<Order>(14); Assert.AreEqual(14, order.Id); foreach (OrderItem item in order.OrderItems) { Console.WriteLine(item.Id.ToString()); Product prod = item.Product; if (prod != null) Console.WriteLine(prod.ProductName); } Assert.AreEqual(2, order.OrderItems.Count); }
Neo4jD–.NET client for Neo4j Graph Database Part 3
This post talks about Index and Graph traversal functionalities added to Neo4jD.
Neo4j supports Cypher, Germlin and REST based api’s for traversal. As of now Neo4jD can creates query for Germlin and REST based traversal. Traversal is not fully implemented in Neo4jD, it’s still in progress.
How to Create Index
Creating index in Neo4j using Neo4jD is very simple as shown below
[TestCase] public void Create_Index() { Index test = Index.Create("TestIndex"); Index fav = Index.Create("Favorites"); }
Indexing nodes with key/value pair helps to search it faster. Let’s see how to add nodes to the Favorites index.
[TestCase] public void AddNodeToFavorites() { Index fav = Index.Get("Favorites"); Node node = Node.Get(1); fav.Add(node, "FirstName", "Sony"); Node node1 = Node.Get(2); fav.Add(node1, "FirstName", "Viji"); }
We added two nodes to the index Favorites, as I said using index we can easily retrieve Nodes using the Key/Value pair. Let’s see how we can query the index using Neo4jD.
[TestCase] public void Search_Index() { Index fav = Index.Get("Favorites"); IndexQuery qry = new IndexQuery(); qry.GetKey("FirstName").StartsWith("Vi").OR().GetKey("FirstName").Equals("Sony"); IList<Node> nodes= fav.Search(qry); Assert.AreEqual(2, nodes.Count); }
You decided to remove a Node from the Favorites index, it’s a very simple call as shown below
[TestCase] public void Remove_Node_FromIndex() { Index fav = Index.Get("Favorites"); Node node = Node.Get(1); fav.RemoveNode(node); }
I removed Node Sony from the index.
That’s all about Index, let’s go through Germlin and REST Traversal
Germlin Traversal
Germlin is a groovy based Graph traversal language, Neo4j has a Germlin plugin to send Germlin script to Neo4j server.
Let’s see how to create a Germlin query using Neo4jD.
[TestCase] public void Get_Out_Nodes() { GermlinPipe germlinQuery = new GermlinPipe(); germlinQuery.G.V.Out("son"); Node father = Node.Get(1); IList<Node> nodes = father.Filter(germlinQuery); Assert.AreEqual(1, nodes.Count); }
For more Germlin query you can reffer Neo4j API reference.
REST Traversal
For Rest traversal we need to provide Json structured query to the server as shown below. For more details goto Neo4j REST API reference.
{
“order” : “breadth_first”,
“return_filter” : {
“body” : “position.endNode().getProperty(‘name’).toLowerCase().contains(‘t’)”,
“language” : “javascript”
},
“prune_evaluator” : {
“body” : “position.length() > 10”,
“language” : “javascript”
},
“uniqueness” : “node_global”,
“relationships” : [ {
“direction” : “all”,
“type” : “knows”
}, {
“direction” : “all”,
“type” : “loves”
} ],
“max_depth” : 3
}
To create the syntax Neo4jD uses a fluent API as shown below.
[TestCase] public void REST_Traversal_Test() { Node node = Node.Get(19); Assert.IsNotNull(node); RestTraversal r = new RestTraversal(); r.Order(OrderType.breadth_first) .Filter(new PropertyFilter().SetPropertyName("FirstName").Contains("Viji")) .RelationShips(RelationshipDirection.out_direction, "wife") .RelationShips(RelationshipDirection.all_direction, "family") .Uniqueness(UniquenessType.node_global) .MaxDepth(2); IList<Node> nodes = node.Filter(r); Assert.AreEqual (1, nodes.Count); }
Neo4jD–.NET client for Neo4j Graph Database Part 2
This post talks about some of the new functionalities added to Neo4jD in recent days.
Persist Static typed object
In the previous post I explained how to create Node and Relationship using Neo4jD. As you might noticed that Node’s properties are dynamic, for e.g to set FirstName we say ‘node.SetProperty(“FirstName”,”Sony”)’. Just like me most of us are not comfortable with Dynamic properties, that’s why we create static typed entities. This post deals with how to persist static typed entities to Neo4j.
Note: I am not against Dynamics, it’s just a personal preference not to build around Dynamics.
Let’s talk with some examples.
public class Person { public Person() { this.Address = new Address(); } [EntityId] public int Id { get; set; } public string FirstName { get; set; } public string LastName { get; set; } public Address Address { get; set; } } public class Address { [EntityId] public int AddressId { get; set; } public string Address1 { get; set; } public string City { get; set; } }
So I have two classes to hold my data. Let’s see how to persist the object using Neo4jD. The core of Neo4jD is based on Node and Relationship with dynamic properties. On top of the core I added a mapper to do the mapping of Static typed entities to Nodes or relationship. I am not going to much insights of mapper, will just go through some e.g., if you need more insight better go through the Neo4jD in github.
How to Save object
[TestCase] public Person SaveTest() { Person person = new Person { FirstName = "Sony", LastName= "Arouje"}; NodeMapper mapper = new NodeMapper(); person=mapper.Save<Person>(person); Console.WriteLine("Generated Id: " + person.Id.ToString()); Assert.AreNotEqual(0, person.Id); return person; }
Behind the scenes the NodeMapper uses reflection to get the data from the Person object and create Node object. Let’s see how I am doing it.
public T Save<T>(T entity) where T:class { Node node = this.CreateNode<T>(entity); node.Create(); return MapperHelper.SetIdentity<T>(entity, node.Id); } private Node CreateNode<T>(T entity) where T:class { Node node = new Node(); node.AddProperty("clazz", typeof(T).ToString()); typeof(T).GetProperties().Where(pr => pr.CanRead && MapperHelper.IsAnId(pr) == false)
.ToList().ForEach(property =>
{
if(MapperHelper.IsPrimitive(property.PropertyType))
node.AddProperty(property.Name, property.GetValue(entity, null).ToString());
});
return node;
}
CreateNode function does the mapping. It Iterates through the property and call node.AddProperty(property.Name, property.GetValue()), it’s same as node.AddProperty(“FirstName”,”Sony”).
You might have noticed that the Id field in the class is decorated with ‘EntityId’ attribute. It’s a mandatory attribute to Get or Set the node id to Entity’s id field.
Get Instance from Neo4J
[TestCase] public void GetPersonById() { NodeMapper mapper = new NodeMapper(); Person person = mapper.Get<Person>(7); Console.WriteLine("Generated Id: " + person.Id.ToString()); Assert.AreNotEqual(0, person.Id); Assert.AreEqual(“Sony”, person.FirstName); }
To get the Person I called mapper.Get with unique id of the person and the mapper will return a valid person if it exist in db. NodeMapper uses below function to generate the object.
public T Get<T>(int id) where T:class { Node node = Node.Get(id); T entity = (T)Activator.CreateInstance(typeof(T)); if (node.GetProperty("clazz")!=typeof(T).ToString()) throw new InvalidCastException(string.Format("Retrieved object with ID '{0}' is
an instance of '{1}' and unable to cast it to '{2}'", id.ToString(),
node.GetProperty("clazz"), typeof(T).ToString())); typeof(T).GetProperties().Where(pr => pr.CanRead && MapperHelper.IsAnId(pr) == false)
.ToList().ForEach(property =>
{
property.SetValue(entity, MapperHelper.CastPropertyValue(property,
node.GetProperty(property.Name)), null); }); entity = MapperHelper.SetIdentity<T>(entity, id); return entity; }
Create Relationships
You can see that Person has an instance of Address, in Graph Person is related to Address. Let’s see how we can create that
[TestCase] public void CanCreateRelationships() { Person person = new Person { FirsName = "Sony", LastName = "Arouje" }; person.Address.Address1 = "EcoSpace"; person.Address.City = "Bangalore"; NodeMapper mapper = new NodeMapper(); mapper.CreateRelationshipTo<Person, Address>(person,person.Address); Console.WriteLine(person.Id.ToString());
Assert.AreEqual(1, person.Id);
}
CreateRelationShipTo function will Save both Person and Address if it’s not saved other wise it’s create a relationship. In Neo4j the relationship will be person->address.
Get Person with Relationship
[TestCase] public void CanGetRelatedNodes() { NodeMapper mapper = new NodeMapper(); Person person = mapper.Get<Person>(12); IList<Address> address = mapper.GetRelatedEntities<Person, Address>(person, typeof(Address)); Assert.AreEqual(1, address.Count); Assert.AreEqual("EcoSpace", address[0].Address1); }
I know the persisting and retrieving relationships are bit messy, I will work on it to make it more clear and silent.
That’s all about persisting Static typed objects to Neo4j.
In the next post I will talk about Creating Index, Traversal using Germlin and REST.
Neo4jD–.NET client for Neo4j Graph DB
Last couple of days I was working on a small light weight .NET client for Neo4j. The client framework is still in progress. This post gives some existing Api’s in Neo4jD to perform basic graph operations. In Neo4j two main entities are Nodes and Relationships. So my initial focus for the client library is to deal with Node and Relationship. The communication between client and Neo4j server is in REST Api’s and the response from the server is in json format.
Let’s go through some of the Neo4j REST Api’s and the equivalent api’s in Neo4jD, you can see more details of Neo4j RestAPi’s here.
The below table will show how to call Neo4j REST Api directly from an application and the right hand will show how to do the same operation using Neo4jD client.
| Neo4j Api | Equivalent Neo4jD Api |
| Create Node | |
|
POST http://localhost:7474/db/data/node Response (click here to see full response) { } |
Node sony=new Node(); sony.AddProperty(“FirstName”,”Sony”); sony.Create(); |
| Get Node by ID | |
GET http://localhost:7474/db/data/node/3
|
Node sony=Node.Get(1); Assert.AreEqual(“Sony”, sony.GetProperty(“FirstName”); The Neo4jD will create a Rest request to fetch the Node with Id 1 and parse the json response and set the properties and required field. |
| Create Relationship between two Nodes | |
|
POST http://localhost:7474/db/data/node/1/relationships Response |
Node sony=Node.Get(1); //Get an existing node //Create a new node Node viji=new Node(); viji.AddProperty(“FirstName”,”Viji”); viji.Create(); RelationShip relation=sony.CreateRelationshipTo(viji, “wife”); |
Client lib is still missing some basic features and I am working on it.
The next important part of the library is Graph traversing. Neo4j supports REST api, Cypher and Germlin for Graph traversing. I am currently writing a query creator for Germlin. So the initial version of Neo4jD will only support Germlin.
If any one wants to join the project feels free to send me a personal mail or add a comment. You all can have a look at the code in Git.
Neo4jD Part 2 Neo4jD Part 3 How to Persist Static typed Entities
2011 in review
The WordPress.com stats helper monkeys prepared a 2011 annual report for this blog.
Here’s an excerpt:
The concert hall at the Syndey Opera House holds 2,700 people. This blog was viewed about 46,000 times in 2011. If it were a concert at Sydney Opera House, it would take about 17 sold-out performances for that many people to see it.
Multi client Asynchronous TCP Server
This post is based on a tracer I am currently working on. My current project requires a TCP server implementation where other client could establish communication and perform data processing. The server should be able to handle multiple request from client. I never did any work with TCP Listener, initial one hour struggled a lot to get it working.
First approach I took was kind of Server – Server communication. That means there is no client both system act as server, it’s bit complex but I created a working prototype in next couple of hours. Yes it is complex, client app needs lot of work to establish a connection.
I need the server more simple and less code for the client to establish connection. So I started searching for simple TCP based client server approach. I stumble upon this post, where the author does a multi client communication. I got a start for my second tracer, a true client server model. I modified it and removed looping construct and used Async model instead. You all can see the code below.
TCP Server
private static TcpListener _listener; public static void StartServer() { System.Net.IPAddress localIPAddress = System.Net.IPAddress.Parse("10.91.173.201"); IPEndPoint ipLocal = new IPEndPoint(localIPAddress, 8888); _listener = new TcpListener(ipLocal); _listener.Start(); WaitForClientConnect(); } private static void WaitForClientConnect() { object obj = new object(); _listener.BeginAcceptTcpClient(new System.AsyncCallback(OnClientConnect), obj); } private static void OnClientConnect(IAsyncResult asyn) { try { TcpClient clientSocket = default(TcpClient); clientSocket = _listener.EndAcceptTcpClient(asyn); HandleClientRequest clientReq = new HandleClientRequest(clientSocket); clientReq.StartClient(); } catch (Exception se) { throw; } WaitForClientConnect(); } }
Here I use a async wait approach rather than waiting in a loop. So whenever a client connect OnClientConnected will fire and it handover the request to Client Request handler. After that it will again go back to wait mode for the next request. The advantage of this approach, the application is responsive and less CPU intensive compare to the looping approach.
Let’s see the client request handler
public class HandleClientRequest { TcpClient _clientSocket; NetworkStream _networkStream = null; public HandleClientRequest(TcpClient clientConnected) { this._clientSocket = clientConnected; } public void StartClient() { _networkStream = _clientSocket.GetStream(); WaitForRequest(); } public void WaitForRequest() { byte[] buffer = new byte[_clientSocket.ReceiveBufferSize]; _networkStream.BeginRead(buffer, 0, buffer.Length, ReadCallback, buffer); } private void ReadCallback(IAsyncResult result) { NetworkStream networkStream = _clientSocket.GetStream(); try { int read = networkStream.EndRead(result); if (read == 0) { _networkStream.Close(); _clientSocket.Close(); return; } byte[] buffer = result.AsyncState as byte[]; string data = Encoding.Default.GetString(buffer, 0, read); //do the job with the data here //send the data back to client. Byte[] sendBytes = Encoding.ASCII.GetBytes("Processed " + data); networkStream.Write(sendBytes, 0, sendBytes.Length); networkStream.Flush(); } catch (Exception ex) { throw; } this.WaitForRequest(); }
}
Here also I converted the waiting in loop to Async wait model. The looping model causes a constant hike in CPU utilization. This post explain how to asynchronously read request from client.
That’s all the server side. The client side is shown below
public class TCPClient { System.Net.Sockets.TcpClient clientSocket = new System.Net.Sockets.TcpClient(); NetworkStream serverStream; public void ConnectToServer() { clientSocket.Connect("10.91.173.201", 8888); } public void SendData(string dataTosend) { if (string.IsNullOrEmpty(dataTosend)) return; NetworkStream serverStream = clientSocket.GetStream(); byte[] outStream = System.Text.Encoding.ASCII.GetBytes(dataTosend); serverStream.Write(outStream, 0, outStream.Length); serverStream.Flush(); } public void CloseConnection() { clientSocket.Close(); }
public string ReceiveData() { StringBuilder message = new StringBuilder(); NetworkStream serverStream = clientSocket.GetStream(); serverStream.ReadTimeout = 100; //the loop should continue until no dataavailable to read and message string is filled. //if data is not available and message is empty then the loop should continue, until //data is available and message is filled. while (true) { if (serverStream.DataAvailable) { int read = serverStream.ReadByte(); if (read > 0) message.Append((char)read); else break; } else if (message.ToString().Length > 0) break; } return message.ToString(); } }
As you can see most of the function is very simple to understand except ReceiveData(). I haven’t concentrated much on the client side as client of the actual TCP server will not be a .NET one. I am not discarding the fact, I will refactor ReceiveData().
Decouple Data interchange format from Domain Model
I was doing a client app to display some data, the client receive data in xml format. One design constrain for me was, consumer’s domain model should have less impact even if the interchange format changes. So once the domain is loosely coupled, it can work on any type of data. The data can be in XML, JSON, or what ever it is. This post explains how I achieved it, it’s a very simple approach based on Strategy and Iterator Pattern.
For simplicity, I read the xml data from a file. Below is the xml
<Library> <Books> <Book Name="CLR via C#" Price="30"></Book> <Book Name="Algorithm Design Manual" Price="40"></Book> </Books> </Library>
Parsers
I created a parser class to parse the xml files. Below is the contract and the parser.
public interface IBookDetailsParser:System.Collections.IEnumerator { string getName(); double getPrice(); }
public class BookDetailsXMLParser:IBookDetailsParser { private int _nodeCount; private int _position; private XmlDocument _bookDetails; public BookDetailsXMLParser(XmlDocument bookDetails) { this._bookDetails = bookDetails; this._position = -1; this._nodeCount = _bookDetails.SelectNodes("/Library/Books/Book").Count; } public string getName() { return this.getValue("Name"); } public double getPrice() { return Convert.ToDouble(this.getValue("Price")); } private string getValue(string attributeName) { var node = _bookDetails.SelectSingleNode("/Library/Books/Book[" + (_position + 1).ToString() + "]/@" + attributeName); if (node == null) return string.Empty; return node.Value; } public object Current { get { return _bookDetails.SelectSingleNode("/Library/Books/Book[" + (_position + 1).ToString() + "]"); } } public bool MoveNext() { _position++; return (_position < _nodeCount); } public void Reset() { _position = -1; } }
As you can see the parser is based on the enumerator pattern.
Domain Model
public class Book { public Book(string name,double price) { this.Name = name; this.Price = price; } public string Name { get; private set; } public double Price { get; private set; } }
public class BookList { private IBookDetailsParser _bookDetailParser; public BookList(IBookDetailsParser bookDetailParser) { this._bookDetailParser = bookDetailParser; } public IList<Book> GetBooks() { IList<Book> books = new List<Book>(); while (_bookDetailParser.MoveNext()) { Book book = new Book(_bookDetailParser.getName(), _bookDetailParser.getPrice()); books.Add(book); } return books; } }
BookList is not aware of what kind of data format it operates on. In case of XML data we can inject the xml parser provided above. Later if we want to change to JSON format then write a json parser implemented by IBookDetailsParser and pass the new json parser to BookList. In this approach any change in data interchange format wont make any impact to our model.
BookService combine all these things together.
public class BookService { public IList<Book> GetAllBooks() { ServerDataReader dataReader=new ServerDataReader(); System.Xml.XmlDocument booksXML = dataReader.GetBooks(); BookList bookList = new BookList(new BookDetailsXMLParser(booksXML)); return bookList.GetBooks(); } }
Any change in the interchange format will affect the BookService. We can make it loosely coupled here as well but that’s not the intention of this post, here I just want to show a loosely coupled domain model from data format.

