Individual posts are now pingback-enabled. If you’re using a pingback-enabled client, can you try a pingback for this entry? I’ve tested it with the test blog and all seems to be well, but there’s no substitute for real clients.
I borrowed Sam Ruby’s idea and included the pingbacks with the comments. Thanks Sam!
I forgot to mention a couple of things – and since I don’t have anything in yet to edit existing posts, it’s gonna have to be a new one. :-)
Many thanks to Brad Wilson, who unknowingly gave me some ideas related to archiving and comments, and whose site provided a basis to learn CSS. My new site uses CSS for styling and most of the positioning, with the single exception of the table at the top for the graphics layout. This is my first heavy use of CSS – hopefully I didn’t screw anything up too bad!
And also thanks to Sam Ruby and Mark Pilgrim for their most excellent RSS validator, which has proven immensely valuable when generating my RSS feed!
Here’s the rundown on the new weblog. Although it bears a striking resemblance to the old version, it’s been completely redesigned and rewritten.
All posts are now stored in SQL Server, and pages are generated on the fly. On the main page, each post now has a title/link, along with links to comments and referrers for that particular post, with counts for each.
Comments are now allowed for each post, and are shown on the permanent link page for the post. Referrers for the particular post are also shown.
The calendar from Radio is gone, and is replaced by an archive list. Far easier to use, and much easier to find a particular post, IMHO.
All posts are searchable, using the search button in the top right corner. This will search only post data, ignoring page-template stuff (like the blogroll), which isn’t possible with Google.
To preserve existing links to old radio-generated pages (links like http://www.rassoc.com/gregr/weblog/2002/11/21.html), redirect pages are in place. These currently don’t auto-redirect, but may be changed to do so in the future. The cool part about these is that they are auto-generated on the server from the admin site, so they can be easily modified en masse.
If you notice any problems, please let me know! Also, subscribers please update the RSS location to http://www.rassoc.com/gregr/weblog/rss.aspx.
After yet another hour of screwing around with Radio trying to get it to upstream my archives, and after yet more crashes when trying to close it, I’ve finally replaced Radio with my own weblog application.
I’ll post more details soon…but for now, I just wanted to explain why your news aggregator is probably showing my last couple of weeks worth of posts (new generated RSS file). If you’re subscribed, please change the RSS location to be http://www.rassoc.com/gregr/weblog/rss.aspx. The old feed will be removed at some point in the future. Thanks!
Ok, every now and then I search for ‘Greg’ in Google, just to see where I am. Today I’m up to #40, behind sites like Dharma & Greg and Greg the Bunny. :-) But the one that caught my eye was #43, which was for Greg Ray. He is an IRL driver (a pretty famous one), and in his bio it says:
First racing experience
Drove a spec racer at SCCA driver’s school, Denver, Colorado in September 1991.
He raced with LaRue Motorsports, which is the company I currently race with. The LaRue’s have two daughters, named Nikki (now 21) and Michelle (16), who work on the cars along with their parents. They have a picture on their site of Nikki and Michelle with Greg Ray, way back when he was racing the Spec Racer Fords. Very cool! Small world. :-)
After writing a bit the other day on server farm caching, I noticed an article in December’s MSDN Magazine titled “Use Data Caching Techniques to Boost Performance and Ensure Synchronization for Your Web Services”. I read it this morning over a bagel…
Basically the article describes a N x N cross-linked notification mechanism for cache coherency notifications. Whenever a server updates the data, it calls web services on all the other servers in the farm telling them to update themselves. It basically looks up the server list in a database, and walks through the list, calling the “refresh” web service on each of the servers.
I don’t mean to be negative, but I don’t think this is anywhere close to something that could be usable for a production server environment. Here’s why:
And in addition, this solution suffers from the problems discussed in my previous post. It may seem like this is all nitpicky, but these are the things you have to worry about when building a system which must run reliably and ensure data consistency in the face of multiple failure modes.
I’m looking forward to seeing what Justin has come up with – this is an interesting problem, without a simple answer!
I’ve been reading about what sounds like a super-cool product – Connectix Virtual PC. It’s similar to VMWare, in that you can create virtual machines and install different operating systems. Great tools – especially for those who must develop and test software for multiple platforms. But what really caught my eye was this feature:
“Undoable Drives allow you to store or discard changes without affecting original state of underlying drive”
Among other things, what an awesome tool to test application installations. I can create an image with, say, Windows .NET Server, and save it. I can then start this OS, run my installation package, see how it went, and undo the whole thing automatically to restore the previous image. Instantly. No re-imaging of a drive. No running the un-install and assuming it worked like it was supposed to. We’ve all been through the pain of having to test and re-test installations on clean OS installations; the pain is even worse sometimes for server installations, where you are running scripts to modify the IIS metabase, etc.
Very cool, if it does what it claims. Was I the only one who didn’t know about this?
Justin is discussing caching in a multi-server environment, which I wanted to comment on.
First, there is the question of what data is good to cache. Justin mentions a possible rule of thumb:
Let’s assume you have some data that is updated 1 time out of every 1000 reads. That’s a good candidate for caching in my mind.
I think this really depends on the application. The more the data gets updated, the more cache coherency processing will be involved (unless you allow stale data, which we will discount for this discussion). And as he mentions elsewhere, the cost of reading the data from the actual data store certainly factors into the equation.
He then discusses the interesting question of how to implement a cache in a multi-server environment. He concludes that a N x N cross-linked notification mechanism is unworkable in real life (I agree in general), and continues:
So what are some other ways? I could do a reliable TCP multicast to all the web servers. That way all I have to know is one IP address. But TCP multicasting doesn’t scale that well in my experience with SwiftMQ and iBus JMS solutions.
The tough problem I see here, even if you could make this work and scale, is the race conditions. Sometimes you use transactional reads from databases to ensure that you read the data either before or after a writing transaction, but not during. But in any case, you know the data is the most up to date available. With a system like this, you could have just updated the cache on your machine, and you’re starting to broadcast updates to the other servers, but at the same time another server is reading the same value from its own cache. The data is stale, if only by a few milliseconds. But in many applications, this will be a huge problem.
So one might say, “I’m only caching data that doesn’t change very often, like my catalog and prices, so this won’t be a problem.” Well, I’m not so sure…if you update the price of an item, but there is a small window where your servers do not agree what the price is for the item, then you’re screwed – and depending on how your load is distributed and how your application is designed, it’s conceivable you could end up with corrupt data.
Justin then describes another scenario:
One thing that I’ve been playing around with is this –
Here I have a cache server. The web servers simply talk to the cache server and the cache server can batch up commands to the database. This works very well with Domain Models. I’ve never tried it using a Table Module Gateway. But it should work the same.
This certainly solves the problem I was just discussing. However, you now have something between the web servers and the database, which has become a single point of failure.
Aha, one thinks, I will have multiple cache servers, and they will somehow ensure coherency between themselves. Well, now you’re back to the original problem of coherency and timing, as well as a fail-over model which maintains absolute coherency – and it’s not an easy problem to solve.
In general, I think this is a hard problem to solve in the general case, and that’s probably why there aren’t any magic cache servers being sold. But for a particular application, it’s not quite as bad, because you know how often your data will be updated, possibly the time of day it will be updated, etc. The easiest data to cache is going to be data that doesn’t change except on a fixed schedule (for example, update the product catalog once a day). For data that is subject to updates at any point, I think you’re left with an application-specific solution, which can take into account the strict-ness of your coherency requirements.
Lots of comments on my previous post. Ingo says:
But what happens, if you later decide that you need distributed TX, probably because another method wants to integrate the addition of a new customer with a post to a message queue? This wouldn’t be possible using the code you’ve shown.
Right, obviously my example only works with a single RM, and it was intended as such. The situation you describe is, in my mind, the most pressing argument used to just bite the bullet with automatic/distributed transactions up front, even if you don’t need them immediately.
What are you doing with that method signature when you’re porting that app to another data provider? What if a future revision of that method wants to add that customer to the database asynchronously and does so by stuffing it into a transactional queue, first?
I read two parts to this. First, the transaction queue scenario is as Ingo mentioned above, to which I concede this implementation is for a single RM only. Second, the oft-pondered question – what if I need to change databases? Well, there are so many more issues involved in changing a database out from under a real-world application, that I don’t give this a lot of weight. The way to squeeze the absolute best performance out of your database is to leverage the proprietary extensions available in that database; for example, using stored procedures. And as soon as I do that, I’ve got some real work to do if I want to switch databases. Another issue is lock management – if you’re changing, for example, between MS SQL Server (lock-based) and Oracle (version-based, last I checked), you need to at least evaluate your transaction locking to make sure you’re getting the right result with the required isolation.
All that said, however, you could relatively easily change my signature and implementation with something that uses a generic interface (IMyTransaction, IMyConnection, etc.) and be able to switch data providers from under it.
On Clemens’ comments page, Tomas doesn’t like my method signature and implementation:
I agree this isn’t a great way of doing it. For one thing, it’s way to intrusive on your objects and methods, and hard to extend.
I could change the syntax to make this look a whole lot more like automatic transactions. A method attribute, combined with a single-call hook in each method (or use the undocumented and soon-to-be-not-working-in-1.1 ContextAttribute and associated interception architecture), combined with implicitly “flowing” a transaction through CallContext, would reduce a lot of the requirements. The example was kept simple to illustrate a point. We’re talking about syntactic sugar.
Morten Abrahamsen has a few interesting comments, which I’ve quoted some snippets from here:
Suddenly a new component requires a new execution context (another process ?) and you get a lot of not so fun cross process marshalling to get the SqlTransaction to interact with your component code.
If this scenario were real, I’d agree, and use a distributed transaction in that case. However, how often (in real life) do I really need a transaction to cross execution context boundaries (we’re still talking about a single RM scenario here)? In my experience, it’s not all that common. And as soon as you do, and start stretching arbitrary transactions across process/machine boundaries, you’re prone to “accidentally” holding database locks for quite some time.
The fear of distributed transactions is in my experience often driven by the fear of performance losses. However, today the performance loss isn’t all that big, yet it is there!
If it were all about execution performance, then I’d be all for distributed transactions. On a typical request, I don’t mind paying a small performance penalty for convenience, and I can add cheap hardware to my business tier to make up for this (to a point, then I’m concerned about performance again). But using a distributed transaction, and an out-of-process transaction coordinator (such as MSDTC), means there is more communication going on, transactions are lasting longer, and thus database locks are being held longer than necessary. And when you’re hitting a wall in database throughput, it’s all about releasing locks sooner (assuming your bottleneck isn’t CPU or IO).
On another note, I always find it interesting when people automatically bind the need for EnterpriseServices with distribution transactions.
Well, I think for most applications, transaction management is really the only COM+ service being intentionally used. Not all applications – but most.
I feel that there is a big difference between a component that accepts a SqlTransaction object as the first method parameter and a “transactional component”. It’s a conceptual difference, a design difference and an implementation difference.
I don’t understand this comment. A “transactional component” is one that can/must do its work in the context of a transaction, and affect the outcome of said transaction. How is the SqlTransaction version and the “transactional component” different?
Ingo also points to a great article that mentions:
In the future, Enterprise Services will support the concept of promotable transactions which will begin as a local transaction and will be promoted to a distributed transaction when required.
NOW we’re talking. That would be awesome.
All in all, don’t get me wrong – I like EnterpriseServices. I’m certainly not knocking the power of distributed transactions, and the convenience of declarative transaction boundaries. In an application I’m working on right now, I may actually end up switching to using distributed transactions to be able to support MSMQ/SQL transactions together – and let me tell you, I’m going to get to delete a lot of code if I do this. But for many applications, which use a single RM, I don’t think using distributed transactions should be the automatic choice; people should evaluate their needs, and intelligently choose a transaction management strategy. Local transactions are lean, mean, and effective. Let’s not dismiss them out of hand.
Ok, I’m going to barge in on this discussion that Clemens kicked off. Hope no one minds. :-) First, my general opinion is that if you need distributed transactions, object pooling, or a consolidated process model (for caching, etc.), then sure, EnterpriseServices is an easy way to get these features. However, I hold that there is a huge class of applications that don’t actually require any of these.
I wanted to specifically comment on something Ingo Rammer said about transactions. To start it off, John Lam wrote:
If I’m writing a managed application that has a web-based front end, and I don’t need distributed transactions, why do I need to care about Enterprise Services?
To which Ingo replied:
The second scenario in which I’d think about using EnterpriseServices is if I don’t know the exact call chain of my infrastructure components. Using imperative, single resource manager transactions absolutely requires you to know the very call chain/s which will be used to call your infrastructure components.
<example type=”very contrived”>
public void StoreCustomer(Customer cust);
public void AddOrder(Order ord);
</example>[…] So, at least for me, the question which should be used to choose between declarative and imperative TX (and therefore also in favor or against EnterpriseServices) is not only “Do I need distributed transactions?” but also “Do I need transactions for which I don’t know the exact call chains which will be used to access my infrastructure layer components?” and “Will these call chains ever change or could there be new ones?”.
So he’s saying that in order to design a robust application with local non-declarative transactions, one needs to know every possible permutation of the call chain. I beg to differ on this; a model I’ve used is as follows. A (possibly optional) transaction parameter can be passed throughout the call chain (you could even make this implicit via the CallContext, but I haven’t seen the need yet). Each function operates based on the passed in transaction. So the outside controller looks something like:
using (SqlConnection conn = new SqlConnection(connectionStr))
{
conn.Open();
SqlTransaction trans = conn.BeginTransaction();
try
{
int custId = Customer.AddCustomer(custInfo, trans);
Order.AddOrder(custId, orderInfo, trans);
trans.Commit();
}
catch (MyException e)
{
trans.Rollback();
// log error, throw exception, etc.
}
}
and the lower level objects look something like:
public int AddCustomer(CustInfo custInfo, SqlTransaction trans)
{
int custId = 0;
SqlConnection conn = null;
if (trans != null)
conn = trans.Connection;
else
{
conn = new SqlConnection(…);
conn.Open();
}
try
{
SqlCommand cmd = new SqlCommand(…, conn, trans);
…
}
finally
{
if (trans == null)
conn.Dispose();
}
return custId;
}
So they can operate in the presence of a transaction, or in the absense of one. If there is not a transaction passed in, they would have the option of creating one, executing without one, or failing. And if all critical (ie. transaction must be aborted) manifest themselves as exceptions, then the transaction outcome will be correct.
The benefit, obviously, is not having to use distributed transactions, and the associated overhead that comes with them. In my opinion, the code above is not complicated, and other than the boilerplate code, the “way of thinking” is much the same as it is with using declarative transactions with EnterpriseServices. And to come back to Ingo’s point, I think if all of your business methods are designed like this, you can add/remove them at will from the call chain, and everything should still work.
