RSS “security”

I was on a call the other day with some folks in the industry, and someone made a comment to the effect of “we really need to come up with some kind of solution for securing RSS feeds – then we can really do some cool stuff.” Before I could get on my soapbox, someone else on the call concurred with the first person.  When I mentioned that this stuff has been figured out already, and started describing the existing widely-used mechanisms, they were both a bit surprised, and suggested I write something about it.  So here we go. :-)

There has been much talk about “RSS security”. The problem is, that’s such a vague term, and you can’t really make much progress until you actually break that down into what you really mean. So let’s give that a shot.

When most folks are talking about security with RSS, they tend to mean three things (or any combination of these):

authentication

authorization

encryption

Let’s take these topics one at a time.

Authentication – this basically means that you want to be able to identify the user who is requesting your feed. There are well-known solutions here – in particular, the tried-and-true HTTP authentication mechanisms, including Basic and Digest. These are the same authentication mechanisms we use on the internet for subscriber-only web sites, and they are equally applicable to RSS.  Behind the firewall, you could add NTLM/Kerberos authentication for Windows networks.

Most mature RSS aggregation tools (NewsGator Online, NewsGator Outlook edition, FeedDemon, NetNewsWire, lots of others) support these widely-accepted authentication mechanisms.

And for those using Windows servers, I’ve even written some sample code for .NET that implements these protocols on the server side, without using the built-in IIS implementation with Active Directory.

Authorization – this means that once you know who the user is, should they be allowed to access the content they are requesting?  This is the easy part…once you’ve authenticated the user requesting the feed, you know who they are, so you can now decide whether they should be allowed to access the content.  Again – just like regular web sites.

Encryption – this means that if someone is watching my network with a network sniffer of some sort, or they’ve managed to insert themselves in the network path between the client and server, then they would be able to see the content going by…so we encrypt it.  The easy way to do this is with SSL – again, exactly how we do it for regular web sites.  And this is usually as easy as adding a certificate to your web server(s), and changing your URL to https.

Piece of cake. Use the mechanisms that have been around for 10+ years securing web content, and you won’t go wrong.

“But wait,” you say. The authentication and encryption mechanisms I mention above are tied to HTTP.  What if you want to transport a feed via some other protocol?

Well then you’d have to think of something else.  You’d likely see what authentication mechanisms are supported by your new transport of choice, and leverage those.  Hmm.

My advice for now?  Don’t worry about it.  RSS today is transported via HTTP.  Sure, you could use other protocols – but almost no one does.  This same argument came up some time ago about SOAP web services…a lot of work went into making sure everything was portable enough to deliver SOAP messages through any arbitrary transport.  But in real life?  Almost no one is doing it.

We don’t need more protocols. We don’t need yet another encryption standard. We don’t need yet another authentication mechanism. Use what works today – it’s proven itself already.

I’m going to write some more about this in the next few days – first, about how an online aggregator should and shouldn’t implement this, and what to watch out for as a user.  There have been some serious problems in the past with flawed implementations of authenticated feed handling, and it’s caused some problems in the community for users…so I’ll write about that as soon as I get a chance.

Spammers vs. Greg – back to even

A few months ago, this blog was crawling with comment spam. Yikes.

Spammers – 1.  Greg – 0.

So around that time, I implemented a CAPTCHA validator here on my blog, so to leave a comment you’d have to actually read the funny letters, and type them in. All was well for a while…this actually eliminated comment spam here for about two weeks.  Woo-hoo!

Spammers – 1.  Greg – 1.

Then the spammers came back, armed with a new resolve.  And my comment feed became useless again, filled with spam before I could delete it.

Spammers – 2.  Greg – 1.

So now, I finally broke down. Comments on posts that are older than 14 days are now moderated. I’m hoping this takes care of most of the problem, since 99% of the comment spam here is on older posts. If the spammers actually take time to read this post and start spamming the recent items, then, well, I’ll have to think of something else.  But in the meantime, there’s been no spam for 30 minutes! :-)

Spammers – 2.  Greg – 2.

Anyway, if you’ve said goodbye to my comments feed in the past due to comment spam, come back!  It should hopefully be much cleaner now. And if you’re using NewsGator Online, here’s a handy link to get it back. :-)

BusinessWeek – Best of the Web

As Steve says, BusinessWeek is looking for votes for the Best of the Web awards.  Be sure to go vote!

BusinessWeek Nominates Blogs, Podcasts for Best of the Web

BusinessWeek magazine has opened its Best of the Web Awards for reader voting. There are ample nominees to choose from, including several of my faves …

For best @Work blog – Robert Scoble among others

For best @home blog – Gawker among others

For best @Home podcasts – IT Conversations and Adam Curry and more

For best Podcasting tool – iTunes, Podcast Alley and more

For best Blog Tools (i.e. aggregators) – Bloglines, Newsgator and more

For best collaboration tool – BackPack (which I nominated), Socialtext and more

NewsGator Online API

It’s been a while in coming…but the NewsGator Online API – the next major piece of the NewsGator platform roadmap – is now online and ready to use.  And you heard it here first…because the press release isn’t going out for another day or two. :-)

Using the NG API, you can build applications that synchronize with our online system, and thus synchronize with other products that are connected to it.  So if someone is using NewsGator Outlook edition at work, FeedDemon 1.6 or NewsGator Online at home, and you write a cool mobile application (for example), they’ll be able to read the same content and sync with your new application.

And this goes far beyond earlier generations of our API, and other folks’ API’s – you can now synchronize not only subscriptions and content, but read/unread state.  Mark an item read (or unread) via the API, and it will be auto-magically marked as read (or unread) in NewsGator Online, NewsGator Outlook edition, FeedDemon, and any other connected product that’s using the API.

This is the same API that our own desktop products (NewsGator Outlook edition 2.5 and FeedDemon 1.6) use, and it’s a subset of the NewsGator Media Platform API.

It’s free for non-commercial use (and licenseable for commercial use); you’ll need to get an API product key (sign in first with your NewsGator Online account), and then you’ll be off and running.  The API documentation is online, and our support team will help you if you run into problems.  We even have a sample application that implements a 3-pane aggregator, with full synchronization support, that one of our engineers wrote in a couple of hours.

We consider the API a work in progress.  It’s solid and reliable – our own products are using it, after all – but we’re definitely looking for any feedback from the community about how we could make it better and easier to use.  So take a look, build some cool stuff, and let us know what you think!

NewsGator Enterprise Server formally announced!

You might not have noticed, if you read my blog often, but we’ve never actually announced NewsGator Enterprise Server.  Well, all that changed today!

It’s officially announced, and will ship in Q3.  Which, if you look at a calendar, is pretty soon. :-)  Here’s the info:

Press Release

Product Info – there are links on this page to the product brochure, white papers, and lots of other cool stuff.

Here’s a comment from one of our beta testers:

I just got beta 2 of Newsgator Enterprise Server setup.  WOW.  Nice stuff.  Honestly I haven’t seen a 1.0 product look this good, well, ever.

We’re pretty excited about it…and now that there is actual information available about the product, I’d love to hear what you think!

NewsGator Enterprise Server performance testing

A couple of weeks ago, we sent a couple of engineers up to the Microsoft Technology Center in Mountain View, to work with some Microsoft guys to do load/performance testing of the upcoming (real soon now) NewsGator Enterprise Server (NGES).
 
So first, a big thanks to Zach and Luis at Microsoft who helped with this all week – they were great.  They had ready-made Exchange environments already set up, and ready to go.  Anywhere from 50 users (the smallest test we did) up to 5000 users.  And load simulators running that would simulate actual user activity for these users, to ensure we were running realistic simulations.
 
We tested the baby-NGES configuration (er, I mean “entry level” – sorry marketing guys!), which runs with the free MSDE SQL engine.  We had projections as to the user counts we could support on this configuration before the connection limit would cripple us…and we were able to adjust those counts.  Turns out we can support more “typical” users than projected.
 
But the real fun started with the large-user-count Exchange scenarios.  One recommended Exchange configuration now is to run large servers with thousands of users on each one.  So in one test configuration, we tested one large 8-way server, with several thousand users.  The following chart shows the CPU load on the Exchange server:
 
 
The orange line shows the time when NGES was switched on in Exchange sync mode.  Variability in the data is mostly due to the (intentional) randomness of the load simulator.  Basically, what this and other data tells us is that there is little effect to the Exchange server of running NGES against it – which soothes a major concern of many IT folks we’ve been talking to.
 
The particular test above was with a single NGES box, with SQL Server on the same machine as NGES.  We have even more data on how many users we support in this configuration, and how many we can support when SQL is moved off-machine.
 
User mix was an interesting question.  We ended up on a mix of:
 
90% of users had 10 feeds
9% had 100 feeds
1% had 1000 feeds
 
And varying percentages of “shared” feeds, where it’s a group subscription rather than an individual subscription, or where multiple people happen to subscribe to the same feed.
 
(aside – we have lots more data than what’s shown above, for hours at a time, and much more than just CPU data…this was just the “prettiest” chart I had handy!)
 
So anyway…like I said, it’s getting close.  The amount of pre-release interest in this product has been amazing to all of us…I can’t wait until the final version ships!

Denver Post News Hound launched

It was announced back in March…and today, the Denver Post News Hound application was launched.  It’s a custom Denver Post application, where their customers can view news and other content from the Denver Post and other publishers (including any RSS feeds).  Here’s a shot of the main window (click for larger):

If you happen to live in Denver, be sure to check out the full-page ad in the paper that will be running for a while.

This is the first public application built on the NewsGator Media Platform…stay tuned for more! :-)

Subscription stuff

I wrote a couple of weeks ago about NewsGator’s subscription plans, and the way our desktop products (NewsGator Outlook edition and FeedDemon) would work with them. Nick Bradbury posted about it a day later. What we were really surprised about is the sheer volume of feedback about this, most of it not-completely-positive. :-)

For those that didn’t notice, Nick posted two days later announcing that we had re-thought this decision. Basically, when you buy NewsGator Outlook edition or FeedDemon, we’re making some changes so those products will continue to work forever, rather than stopping when your subscription runs out.  Certain features (that rely on parts of NewsGator Online) will likely be disabled when your subscription runs out, but the software itself will continue to function.

The plan is to still use the same activation system for this that we have in place already.  So you’ll still need a NewsGator Online account to activate the products…however, even if your account expires you’ll still be able to activate the versions of the products that you had access to originally.  We’re still working out the details, but we’ll definitely let you know as soon as we have something concrete to talk about.

There’s been someone posting a lot of spam comments on blogs and forums about this; for an example, scroll down in the comments on Chris Pirillo’s post here.  You can find this identical comment lots of other places as well, if you look hard enough.  In the middle of that comment is this:

Hi my friend,I want to tell you something about nick’s promising is a lie:
feeddemon won’t work after your subscribtion runs out.

Come on now…no one is lying.  The existing builds of Outlook edition 2.5 and FeedDemon 1.6 will indeed not activate if your subscription runs out…however, we are making changes to both products and the online activation system soon, which will rectify this.  And in the meantime, everyone who is affected by this has plenty of time in their existing subscription, so no one needs to “wait” for us to finish it.

So anyway, I just wanted to publicly clarify what we’re doing, and try to help stamp out some of the FUD that this individual has been trying to spread.

Why subscription software?

As many have noticed, and blogged about, NewsGator’s individual products (meaning everything except NewsGator Enterprise Server) are now all sold on a subscription basis. This includes Outlook edition 2.5, and the upcoming FeedDemon 1.6. Actually Outlook edition 2.0 was sold on a subscription basis starting earlier this year, but we didn’t make a big point about talking about it.

Lots of existing customers have asked why we did this…so let me talk a bit about it. Contrary to popular belief, we didn’t have a secret meeting where we tried to figure out how to squeeze every last dollar or euro out of our customers. :-)

So why, then?

Well, here’s the thing. Back in January of 2004, NewsGator Outlook edition 2.0 was launched, along with NewsGator Online (formerly NewsGator Online Services). Outlook edition had a license fee (it was $29), and if you wanted to use sync or any of our online features, the online subscription started at $5.95/mo.

Later, in October of 2004, we made parts of NewsGator Online (most notably the web edition and media center edition) available for free. Other services (smart feeds, premium content, mobile edition, etc) were still available on a subscription basis.

So far, so good, we thought. We were selling stuff the way people were used to paying for it – that is, some stuff for free, a license fee for software, and a recurring cost for services. Right?

Wrong, as it turns out. Problem is, customers were confused. As we started to build more cool services in NewsGator Online, and as we started making more direct enterprise sales, we realized that everyone was confused. As customers learned what was available with the online system, they wanted it…but the whole license + subscription thing was too complicated.

And when we add new products like FeedDemon in the mix, it gets even more complicated, since FeedDemon will also rely on NewsGator Online features.

So what to do. We could just have a license fee, and eat the service cost. That’s what Intuit does with Quicken, as I recall…but they shut off the services after some period of time for each software release, and they piss off a bunch of customers every time they do it. Also, this doesn’t solve the problem for a user who doesn’t want the Outlook client – we’d be back to a subscription for them.

So we went the other way. Get rid of the license fees, and go with _only_ subscription pricing. Include all of the products in the subscription bundles – including Outlook edition, and FeedDemon (real soon now). And at the same time, we dropped the pricing. Outlook edition comes as part of the cheapest business subscription plan, which starts at $1.95/mo or $19.95/year. This is $10 cheaper than the old license fee.

And, we’re constantly adding value to the subscription bundles. For example – subscribers will get FeedDemon 1.6 as part of their subscription.  No extra license fee, no nothing. So for as low as $1.95/mo or $19.95/yr, you get both Outlook edition and FeedDemon.

“But wait,” you say. “I don’t care about the online system – I just want the products.” Ok – let’s do a quick analysis over 2 years to get both products:

Old pricing: $29 (Outlook) + $29.95 (FeedDemon) = $58.95 (plus upgrade fees)

New pricing: $19.95/yr x 2 years = $39.90

Cheaper after 2 years, and at the end of 3 years, you’d be roughly money-equivalent.

But maybe you didn’t want both products, but really only one of them. The free upgrades are really the advantage then. Another example:

Old pricing:
– initial fee $29
– major upgrade in 1 year $20
total – $49 for two years

New pricing: $19.95/yr x 2 years = $39.90

With the subscription plans, you get free upgrades, new products, and access to the ever-growing services available on the online site. And we’re committed to adding cool new stuff all the time to these plans – witness what we’re doing with FeedDemon. And hey – tell us what YOU want in these subscription plans…we love suggestions.

But what happens when your subscription expires? Well, the products will no longer activate. But you’re not going to be locked out of your data or anything…you just won’t be able to retrieve new content until you re-activate. You’ll have full access to your data even after your subscription expires.
So anyway, that was a bit long-winded. But we really feel like this was the way to go, given the challenges we were facing selling into consumer, business, and enterprise markets all at once.

And I know we’ve probably already lost some existing customers over this. But we’ve tried hard to make this work for you – so riddle me this. For those who purchased Outlook edition 2.0 prior to the subscription change, we’ve offered you 2 years of free service. Try it out – use it for those 2 years. And then come back at the end of that, when your subscription is up for renewal, and tell me honestly if you feel like you’ve gotten your $29 worth – and tell me if you feel like it’s worth $19.95 for one more year.

We’ll listen – I promise.