Well this is a little odd:
I’m twisted. Not as bad as Rory Blyth, but who else has a geek dream with Greg Reinacker (author/owner of NewsGator) playing the part of the future Erik Estrada (from CHiPs)? See for yourself:
[…]
Apparently I’ve parked here for years, but his new wife is none too happy about it. She tells him to get rid of me.
I’m speechless.
In the beginning, there was the Outlook News Aggregator. Not a very imaginative name, but the start to something that would become much more than an Outlook add-in.
About a week later, the name NewsGator was born. I still remember that day…having lunch with a friend, brainstorming about names, and seeing which domains were immediately available. And then ponying up $10 to register the domain, the first expense I can recall that’s attributable to NewsGator. :-)
Later in the year, the questions started about “Gator”. Folks associate this with the evil spyware company now known as Claria, and we started getting questions about whether we were related to them. Of course we are NOT related to them in any way whatsoever, but nevertheless the question comes up constantly.
Last week we had a board meeting, and kicked around the idea (again) of changing our name. There are obviously both advantages and disadvantages to doing so. We’re well known in the space as NewsGator, but on the other hand there are probably sales we’re losing because people assume we’re evil. It’s a tougher decision than one might think, especially at this point in our company life cycle.
So I pose the question to you, my consistently insightful readers. Do you think we should change the company name? Why or why not?
Every couple of months, there is another uprising about the bandwidth usage of RSS…the most recent one has been going on in the last couple of days, and this post from Robert Scoble is right in the middle of it, along with its associated comments. In another post, he even says “RSS is broken.”
As you could probably surmise, NewsGator’s own RSS feeds (such as the News/Updates feed) generate an enormous amount of traffic. This isn’t unexpected, and our network is designed for this…but I understand what people are seeing with their feeds. We use HTTP caching mechanisms to dramatically reduce the total bandwidth requirements, and other internal caching mechanisms to reduce overall server load.
90% of the discussion on the bandwidth issue centers around RSS aggregators, and how they are allegedly abusing servers relentlessly. Robert makes a rough estimation that hits increase 20x by having a RSS feed on a site like MSDN. He also surmises that this will get worse and worse over time:
This gets worse over time because on most sites HTML traffic will go down as people move away (at least until the site reposts interesting content that’ll bring back more traffic) while RSS just grows and grows even if new content doesn’t get posted because people subscribe and don’t move away.
Let’s look at two cases. Let’s make the assumption that the “average” aggregator will default to polling once an hour. Let’s also assume that the server implements HTTP caching headers in some way – really, I consider this a minimum entry criteria for RSS on a busy site.
Case 1 – the content on the site doesn’t update often (let’s say once a day). If the feed only updates once a day, 96% of the requests for the feed (23/24) will return a 304 Not Modified response. The other 4% of requests will respond with the entire contents of the feed. For the 304’s, the bandwidth is small (not negligible on an extremely busy feed, but low enough to not be a huge concern)…total number of connections are something to worry about, but typically not a big issue in most environments.
And the 4% will drop even smaller if the content is updated less often than once a day.
Case 2 – the content on the site is updated often, such that there are almost always changes from hour to hour. Assuming the feed updates in real time, every request to the RSS feed in our example will return the entire feed. This is the case that’s worth worrying about.
Given case 2, there are a number of things that can be done. Fewer items in the feed, excerpts versus full content; all of these have their issues. Some folks have suggested serving incremental content changes based on if-modified-since headers, which not only violates the HTTP specification, but breaks in common caching proxy scenarios. So what can you do?
One possible thing you could do is use caching headers to limit the potential “exposure” of a shorter-than-ideal aggregator polling interval. Nick Bradbury describes one such way to do that here.
Another similar option would be to batch feed updates to once or twice a day. All of the RSS feed requests would return a 304, except for those that occur just after the daily update(s). If there is one update a day, you cut 96% of the required bandwidth in our example. But wait – isn’t the point of RSS to get quick updates to site changes?
Now it gets interesting in a different way.
Back to Robert’s example, he assumes that users without RSS will break down as follows:
20% will visit at least once a day
40% will visit at least once a week
20% will visit at least once a month
20% will not visit in any one month (assuming these folks visited before but just aren’t revisiting)
But look at it this way – 80% of users will be at least a week behind on new content, and 40% will be at least a month behind.
So do you care about these users? Do you have content that you think they would be interested in, if only they knew about it? Would you benefit in some way if these users were reading your content more often? If yes to any of these, RSS helps.
You’re distributing incremental content to users who might be interested. From a business perspective, you can’t compare the bandwidth required by that process to the bandwidth required if these users only occasionally come to your site.
Further, the RSS hits will generally be smaller that the corresponding HTML pages, and also have less ancillary impact (such as images on the site, layout, etc). For example, my weblog front page is 58KB right now, and the RSS feed is 19KB. Adding images and such to the HTML version, and let’s call it 80K, approximately 4x the RSS size.
So I’m finally getting to the point. :-) Assume there is benefit to having users read your content every day. If you had some way to convince your interested users to do this (which of course you’re trying to do), going back to Robert’s example for the HTML site:
1000 users x 30 visits/month = 30,000 visits/mo (assuming once/day)
This is the ideal case for the site – assuming more exposure for your content is better. We’re not counting ancillary hits here, which will certainly add to the server load.
With RSS, let’s say we set it up to update/publish the feed 4x per day – which gives aggregator users an average 3 hour delay before they learn of new content (vs. 24 hours for the HTML):
1000 users x 120 hits/month = 120,000 hits/mo
Remember, all of the other hits (potentially 20 per day per user) are negligible in terms of bandwidth due to cache header implementation.
So we have 4x as many hits, but 1/4 the overall size…so it’s a wash in terms of bandwidth. And users are exposed to your content multiple times per day, which is good for you and them both.
If quicker updates are important for your users, then there is an incremental bandwidth cost to pay for that…but you as the publisher can control this, based on the information you’re trying to push.
Anyway, many of these numbers are pulled out of the air…but the point is, most mature aggregators (like NewsGator and NewsGator Online) use the HTTP caching mechanisms, so use them. And further, there are things you can do on the server side to manage the bandwidth load, depending on the goals you have for your feed.
Comments welcome as always!
In all the excitement of the last couple of weeks, I almost forgot to post about something pretty cool.
SnapStream recently shipped their $49 Firefly remote control…basically you can hook this baby up to your PC, and control it from across the room. I’ve got a couple of them in my office – it’s quite nice, and it’s a RF remote so you don’t have to worry about pointing it the right direction. Combine it with Beyond TV (another of their products), and you’re got a pretty inexpensive but very powerful PVR.
What’s even more interesting, though, is the recent launch of SnapStream Spotlight. This is similar to the “Online Spotlight” feature in Windows XP Media Center Edition, but it ships with the Firefly. And NewsGator Media Center Edition is supported within SnapStream Spotlight.
So why is this cool? Well, Windows XP Media Center Edition used to be the only way to use NewsGator MCE. And it only ships on new PC’s…which means there are millions of folks out there who can’t watch streaming on-demand video (and view any other content) with NewsGator MCE, until they buy a new computer. But with a Firefly, anyone with Windows 2000 or later can use it now. That’s a pretty serious jump in potential market.
We’ve recruited people for some of our open positions, but we still have a few development-related positions available. We’re in the Denver area – local applicants only for now, please!
You’ve almost certainly heard by now that NewsGator Technologies has taken on venture funding from Mobius Venture Capital. I haven’t had a chance to comment about this until now, but there’s quite a bit I want to talk about.
First, to the folks that sent their congratulations, thanks!
Brad Feld, from Mobius, has written a great post describing why they made the investment. Brad and I have been working together on this for a couple of months now, and he’s right on the money in what he says. Go there, and read it now. I’ll wait.
NewsGator for Outlook was our first product, and many of you have been with us from day one. It’s been a great product – tight integration into Outlook, a great story for businesses wanting to leverage RSS, and an amazingly supportive and enthusiastic user base.
Almost exactly one year later, NewsGator Online Services was launched with a number of exciting capabilities. Synchronization between multiple machines running NewsGator for Outlook. NewsGator editions for web browsers, mobile phones, and email clients besides Outlook. Custom search feeds and premium licensed content rounded out the initial services. Media Center support followed a couple of months later. The motto “The content you want. Any time. Any place. Any device.” was a primary driving factor for NGOS, and we’ve been delivering on that ever since.
If there’s a downfall with NGOS, however, it’s been getting the word out about it. We’ve done a good job marketing NewsGator for Outlook, to the point that the brand is widely recognized as a great Outlook product. However, we haven’t yet done a great job of getting the word out about NGOS, and what its capabilities are – especially for the non-Outlook market. We have lots of satisfied customers with the online products, but it’s not yet a “household name”. We’ve been working on this, and we’re getting a bit better…and more changes are underway to further move this along.
The reason I bring all this up is for all the folks who wrote about this, asking “why would Mobius fund an Outlook add-in?” or “Won’t Microsoft build one of these?” or something similar. As Brad said, what we’re doing goes far beyond the Outlook client – to the NGOS capabilities, and some other cool things we’re working on. That said, the Outlook client remains a key part of our strategy, and it will only get better and better.
So this all begs the question that has been subtly and not-so-subtly asked: why outside funding, and why now? NewsGator has been profitable, and doing fine up to now, with several employees and many contractors. So what were we thinking?
As the plan and vision for the products has taken shape, a number of very cool ideas have arisen. Many of these are based on feedback we get from customers, and others are based on feedback from folks who don’t use our products. We’re excited about this stuff – you can see it in our emails, hear it in our voices, and wonder about it when we’re all still online at 11pm. :-)
There are a number of things driving the development schedule. For maximum impact of what we’re working on, we need to do many things simultaneously; and to do that, we need more resources. We could have done all of this without any outside funding, but it would have taken longer. Much longer. And by the time we finished, the market might have been a different place. So we made the decision to accept an investment, and I feel fortunate to have Brad as a partner moving forward.
So there it is. I have lots more to say, but I’m a little afraid I’ll never have time to finish this post if I keep going. :-)
To our existing customers, I’d like to say thank you – you’re the ones that got us to where we are today, and we won’t forget that.
And finally, to anyone who is afraid we’ll take a turn for the worse as we ramp up in size…I don’t think it will be that way at all. But if you see something we’re doing that you don’t like, or you feel like we’re not being fair to our customers, then call us on it. Send me a note. I might not always agree with you, but I promise I will read it.
Gary has cleverly figured out a way to get RSS content from NewsGator Online Services into his new Gmail account:
Now though I’ve started to play with GMAIL, and the first thing I wanted was to use google’s search capability against the many rss feeds I get. It works! I went out and found a pop3 redirector, Mail Redirect and am testing it against the Newsgator online feed. So far, it’s working great, every two hours it goes out, pulls my feeds and redirects them to google.
Great idea – even though Gmail doesn’t support POP3, he’s found a way to send content from the NGOS POP edition to it!
For anyone who’s going to be at Supernova, be sure to check out the “Spam and the Future of Email” panel Thursday at 4:30pm…I’ll be on that panel, joined by some folks from Cloudmark, Turntide, and Oddpost. My contribution to the panel is obviously in the relationship of RSS and email, and how RSS can potentially help with the spam problem for certain categories of email.
And even if you don’t go to that panel, be sure to find me and say hello if you’re at the conference!
…also known as Blogging for Associations, Non-profits & Content-driven Web Sites, is happening on June 30 in Washington, D.C. Debbie Weil, a wizard of online publishing and marketing, has lined up Komra Moriko to speak. Debbie says there are still a few spots left as of today.
NewsGator Technologies is pleased to sponsor the seminar, and is giving away a 6-month subscription to NewsGator Online Services (a $65.70 value) to all attendees.
To quote Debbie – “Blogging for business isn’t about being cool…” Well said!
I’ll be participating in a RSS round-table webcast on Thursday morning, 10:00am PDT. It’s free – if you’d like to listen in, you can register here. On the call will be Mark Fletcher, Mitch Ratcliffe, and Harry Hayes. [the registration page says differently, but I believe this is the more up-to-date information]