Structure Too Big

I think I finally got Peter hooked on WorldMaps. My latest victim. And of course, it wouldn't be possible with the official WorldMaps evangelist, G. Andrew Duthie . Thanks guys!

A few questions came up on Twitter, so I thought I'd take the time to expound on them here. The first question was: who are the top WorldMap users? Here's the top 10 at a glance:

Site	Hits
http://dancesportinfo.net	5,137,942
http://blogs.technet.com/askperf	722,489
http://www.structuretoobig.com	427,112
http://www.irritatedVowel.com	309,104
http://www.wynapse.com	243,283
http://blogs.technet.com/benhunter	155,276
http://timheuer.com/blog/	147,714
http://blogs.msdn.com/gduthie/	121,912
http://blogs.technet.com/davidcervigon	113,178
http://sharepoint.microsoft.com/blogs/GetThePoint/	109,067

So there you have it. I'm going to build a "live" top 50 page soon that include links so you can view the data breakdown a bit more thoroughly.

One question that came up is how my stats (currently #3) are gathered -- for example, do I get a hit for everyone else's hit? A fair question since I'm using my own service.

Looking at the data above, the answer is obviously "no," traffic from other sites doesn't affect my numbers. The sum of all hits is around 8.5 million compared to my ~400k (see an all-user mashup here ). Indirectly, much of the traffic *is* driven from folks clicking through to my site.

One way to measure this is by looking at the Global Domination and Unique Domination statistics on the maps (bottom right corner). Global Domination shows how many unique locations you've hit in contrast to all known unique locations. Remember, though, that this number is relative. For example, as far as I can tell, all locations in Manhattan are considered 1 location. So while you may have thousands or even millions of users visiting from Manhattan, it's resolved as only 1 unique location. Unique Domination is how many of those locations belong _only_ to your map -- locations you are hitting that no one else has. The cool thing about this number (at least I think it's cool) is that it will continually grow smaller. Before long, having ANY unique domination above zero will be a prized value.

Another question is: How is rank determined? First, it's completely possible for some users to be lower in rank yet have stunning World/Unique Domination. Look at Andrew's stats . The poor guy just fell to #8 ... but holy cow, look at the global/unique domination stats. He's schooling me and just about everyone else.

Quite simply, rank is determined by the sum of all hits. And I'm thinking it's time for a change. The question on the table that I ask everyone who uses it is, what is the fair equation for determining rank? Total hits as it stands today? Hits/day average? Unique IPs? Global Domination? Or a mix of all of them?

As for my stats personally, I have an edge. I was using the system months before it was available to the public, and even then, users were very slow to sign up. While my hits/day is lower than many in the top 10, my time on the field has been longer, and has carried me a bit. I admit that. :)

But then, I do only use it on my home page nav, not on every page. For example, I could use the tracking pixel version on my master page so it shows up on every page on my site, and then just display the map on my home page. The end result would be that my Global/Unique Domination and Unique IPs would remain unaffected, however, total hits and hits per day would skyrocket. So you see, there are many ways to interpret the stats and hard to treat it as a strict comparison.

So what's on my to-do list?

1. Top 10/25/50 page, with links.
2. Web services for data portability.
3. Silverlight integration (somehow ... anyone want to help with that?)
4. 1:many accounts or subaccounts.
5. A "no user" page. (If you visit the "all user " page, this would be the opposite of that -- what locations have no one hit yet? Where the all user page has high Global Domination and zero Unique Domination, this page would be the opposite -- zero Global Domination and high Unique Domination.)

posted 04/24/2008 10:32

avg rating: none

comments: 0

trackbacks: 0

Wednesday, April 23, 2008

geekSpeak Today!

Later on today at 3pm, I'll be joining my colleague Glen Gordon on geekSpeak, where we'll be hosting a talk with Karl Shifflet . I'd have to sum up Karl's year as ... "insane." I admit, I barely knew Karl a year ago and now he's everywhere!

Today he'll be talking about the Mole Visualizer tool he has been developing. For more information or to join, go here:

http://msevents.microsoft.com/CUI/EventDetail.aspx?EventID=1032374735&Culture=en-US

posted 04/23/2008 07:41

avg rating: none

comments: 0

trackbacks: 0

Monday, April 21, 2008

Raleigh Launch Event

I'll be presenting at the { Heroes Happen Here } event in Raleigh on May 2nd. It looks like the tracks are all full, but check back at the site if you're waiting to get in! (And hey, if you've registered and don't think you're going to attend, be sure to de-register.) To be honest with you, the content at our launch events is fairly lightweight, so while I'd love to see the room jam-packed, if you're looking for deeper and varied content, keep your eyes out for more info on the Charlotte Code Camp coming in a couple of weeks.

In the meantime, Dug Wilson (newly appointed president of the Triangle .NET User Group ) had a great idea -- a "Heroes" after-event (dinner and a movie). Since Iron Man debuts the same day as the Heroes event, why not check it out on the same day?

See you then!

posted 04/21/2008 09:14

avg rating: none

comments: 0

trackbacks: 0

Monday, April 14, 2008

My Eyes, My Eyes!

I came across this today while redeeming an online coupon. Let's just get right to it without the preamble (click for larger version):

Oh, how this pains me. To protect the innocent I blocked off what would otherwise be incriminating, but suffice it to say this came from a large retailer. And sadly, this is one I've been to -- I don't mean I've been there shopping (though I have), but I mean I've been to their offices, though as far as I know this is outsourced and hosted elsewhere. Let's look over this...

First and perhaps least important, scrambleId is a bigInt or others long numeric field based on the querystring (not visible in the picture, but take my word for it). Yet, its data type is a string (inferred based on Request.QueryString) and it gets passed into the query as a string. But it's not a string. Now you might be thinking: the database column uniqueID may be a varchar or some other string-esque data field, making this "ok." It might be -- but then, not a very good choice for either a primary key or, as labeled, a unique identifier. Design-wise, this is just broken. At the very least, it's confusing. Since I'm picking on it, let's get nitty-gritty and say I'm not a big fan of a column named barcode in a table named barcode.

Second and mildly broken: never output details like this to the client. (OK, this is something I'm guilty of at times -- but then, there's a world of difference between a personal site on a shared host with no information of value in the database vs. an e-commerce site collecting PII.) Handle and log errors, but disable this kind of output via the web.config, especially in a production environment.

Last and obviously the "meat" of the problem: HUGE SQL Injection opportunity! There is NO data validation on scambleID, and it's just handed over to the SqlCommand object. Because another page takes data and presumably writes it to the database, I'm going to make the assumption that this user (based on the connection string -- likely the same one) has write and possibly dbo permissions. I imagine it would be trivial to cause major havoc by dropping tables, updating values, compromising data, etc.

So how is this problem eliminated? Don't execute direct SQL. Abstract that into a stored procedure, and pass in the scrambleID as a parameter. Change the permissions to allow this user execute only permissions on the sproc, not direct table access. If possible, correct the data types. I suspect there may be reasons in play for it being all-numeric (for example: a cash register that may only take numeric values). A bigint can handle over 9,000,000,000,000,000,000 values, so that ought to be big enough. Maybe it is a bigint, and SQL is casting it -- can't really tell. But the nice thing is, if the value must be numeric, it's super-easy to check. Frankly, even if you must use dynamic SQL like this, it would be easy to check the parameters. An Int64.TryParse() or Regex would be easy to implement.

Anyway, this goes to show that these kinds of problems are very real in the world, and it's critical to evaluate them. Peer code reviews, threat modeling, and security education can prevent these!

posted 04/14/2008 16:49

avg rating: 5

comments: 1

trackbacks: 1

Saturday, April 05, 2008

Geek Happy Hour and Launch Event!

It's going to be a busy week! In Raleigh on Monday evening? Join Brad Abrams and me at the Carolina Ale House at Brier Creek. We should be over there by 5:00pm. If you'd like to meet Brad and say hello, stop on over! We're keeping things casual, but fear not: Brad will back in action on Wednesday when he presents at TRINUG .

Sandwiched between the two is the VS2008/WS2008/SQL2008 launch event in Charlotte! Check it out here:

http://www.microsoft.com/heroeshappenhere/events/Charlotte/default.mspx

I'll be presenting on Web Development with VS2008, the first session in the developer track. Most of the event has sold out, but if you're in the area and would like to come, register anyway and come on out. Room just might open up.

posted 04/05/2008 06:18

avg rating: none

comments: 0

trackbacks: 2

Friday, April 04, 2008

Screen Clippings...

Some interesting screen clippings I came across.

The first is of the winning bid in the Charter High Speed Internet auction. The auction, I imagine, took on a life of its own and the winning bid:

Wow! The nice news is that Charter donated the proceeds to charity. As some have pointed out, though, that wasn't announced until last minute, so most of the bids were done not knowing this. Still a good cause, but can't believe the auction got that high. Not sure how high I'd go to get internet for life -- but boy, that's in another league.

Next, this one on the weather.com site and the chance of rain April 5th:

Let me explain this one a little. It might have been Jerry Seinfeld who said something to the effect of disliking the "50% chance of rain" forecast -- "basically you're saying 'maybe it will rain, or then again, maybe it won't.'" But, I don't know if I've ever seen 100% chance of rain. Cool -- we need it. I just don't know if I'm ever 100% sure about anything ... I applaud their certainty. There's just that cynic in me that imagines some weatherman saying, "No matter what happens, it will positively, absolutely rain tomorrow."

posted 04/04/2008 16:05

avg rating: none

comments: 1

trackbacks: 0

Monday, March 31, 2008

Raleigh Silverlight Group

Very cool news from Rob Zelt . The web applications SIG of TRINUG will be doing some Silverlight-focused content. He'll be speaking this Wednesday (4/2). From Rob's blog:

I've been talking with a number of people about starting a Silverlight focused group here in the Raleigh/Durham/Chapel Hill area. If you're interested, se if you can come out to the presentation I'm giving for the Triangle .Net User Group Web Applications SIG on Wednesday and we can maybe chat afterwards as well about setting another meeting up later in April. I'm thinking maybe April 22nd, 24th, or 29th?

If you're interested in Silverlight, be sure to come out.

posted 03/31/2008 19:39

avg rating: none

comments: 0

trackbacks: 0

Thursday, February 28, 2008

TechEd: Birds of a Feather Sessions

Via Rob Zelt !

INETA and Culminis are helping to organize the Birds of a Feather (BoF) sessions at TechEd coming this year in June. If you've never attended a BoF session at a large conference, you really have to go. Some of the best networking opportunities happen at these sessions, so if you'd like to submit suggestions or volunteer to help, check out Rob's post on the subject.

BoF submissions and voting closes on March 19th!

posted 02/28/2008 19:18

avg rating: none

comments: 0

trackbacks: 0

Thursday, February 21, 2008

VS 2008 Install Fests! Register Now!

We had a fantastic run of events with the Visual Studio 2008 Install Fests December and January!

Just a reminder, the registration site for the free copies goes offline tomorrow, February 22nd! If you received a copy of VS 2008 at one of the Install Fests, and haven't registered it yet, please do by EOD tomorrow! (A little birdie told me the site may stay online over the weekend, but still, get it in as soon as possible!)

posted 02/21/2008 10:02

avg rating: none

comments: 0

trackbacks: 0

Monday, February 18, 2008

New Regional Director for the Carolinas

Over the weekend, the Raleigh User Group hosted their third code camp! What a great event. I just returned from TechReady 6 in Seattle (more on that later) and the big news is: Jim Duffy is now a Microsoft Regional Director for the Carolinas! Congrats Jim! See his post here .

As we frequently get asked, just what is a Microsoft Regional Director?

I like Richard Campbell's answer: "We're not a Microsoft employee, we don't necessarily have specific regions, and we don't direct people. But otherwise the name works." To be honest, the "region" part is one of the big factors we added another RD to the area. Many RDs do a significant amount of travel and are effective in their roles from a global perspective, but as a local evangelist, my main concern is the local ecosystem.

But, back to the question. Jonathan Goodyear has a post about it here . I like this summary:

The role of an RD is to act as an unbiased third-party evangelist of Microsoft products and services and to work with software developers to ensure successful project engagements. We act as the glue between Microsoft and the developer community.

Sounds like my job, doesn't it? Except, I'm not an unbiased third-party. I'm a completely biased first party :) ... seriously, one of the biggest assets these guys and gals bring to the role is their perspective. We (as evangelists) partner very closely with the RDs as much of what we're trying to do is equivalent. The good news is,

Jim has been a fantastic resource locally, and we've partnered on a number of events. He's been a long time MVP, and he's also a friend of mine, so I'm pleased to have the opportunity to work with him! Once again, congrats Jim!

posted 02/18/2008 09:23

avg rating: none

comments: 0

trackbacks: 0

Sunday, February 17, 2008

VSTS Briefings

Just saw the following events coming soon to Raleigh and Charlotte (as well as couple of other locations in the southeast). If you're interested in VSTS, check 'em out!

Come join your technology peers to learn about current and future .NET technologies. The focus of this 1 day seminar will be around Microsoft Visual Studio 2008 Team System. You’ll be provided with an overview of each role and the session will wrap up with a preview of the next version of Team System (codenamed “Rosario”).

9:00am Intro Visual Studio Team System/Team Foundation Server Business Value

9:30am VSTS Project Management and Collaboration Features

10:15am Break

10:30am VSTS Tester Features

11:15am Version Control and Build Management

12:00pm Lunch (Provided)

12:30pm VSTS Architect/Developer Professional Features

2:30pm Team System Futures (Rosario+)

3:00pm Closing /Q & A

DATES/LOCATIONS

When: March 18, 2008

Location: Microsoft Office – Charlotte, NC

8055 Microsoft Way
Charlotte North Carolina 28273
United States

REGISTRATION: To Register, click HERE or call 877-673-8368 and reference Event ID:1032367770

------------------------------------------------------

When: March 20, 2008

Location: Microsoft Office – Raleigh, NC

4825 Creekstone Dr., Suite 190
Durham, NC 27703
Phone: (919) 474-4900

REGISTRATION: To Register, click HERE or call 877-673-8368 and reference Event ID:1032367772

------------------------------------------------------

When: March 24, 2008

Location: New Horizons – Birmingham, AL

601 Beacon Parkway West, Suite 106
Birmingham, AL 35209

REGISTRATION: To Register, click HERE or call 877-673-8368 and reference Event ID:1032369273

------------------------------------------------------

When: March 25, 2008

Location: Microsoft Office – Alpharetta, GA

1125 Sanctuary Pkwy., Suite 300
Alpharetta, GA 30004
Phone: (678) 629-5700

REGISTRATION: To Register, click HERE or call 877-673-8368 and reference Event ID:1032367769

posted 02/17/2008 09:36

avg rating: none

comments: 0

trackbacks: 0

Thursday, February 07, 2008

Increasing Scalability in ASP.NET Apps

Time for another series of posts!

This time, I'll talk about a few scalability options in ASP.NET applications. Getting higher performance and greater scalability isn't a one-time, one-shot thing -- rather, it's something that is measured and tweaked continually. While it's true that a good design up front is important, it's also true that you can't focus too much on performance tuning early in the game.

What alarmed me recently was taking a look at my performance stats up until the last few days in January, 2008:

You can clearly see the worsening performance over time. Between March of '07 and Oct '07, my page gen time nearly doubled; between August of '07 and September of '07, my WorldMaps gen time _more_ than doubled (!), RSS and home page gen times were somewhat flat.

Because some of the numbers were fairly flat, I'm inclined to think that nothing profound happened on the server side (this can happen on shared hosting and is sometimes unavoidable). Rather, it's likely some "thermal layer" for traffic was crossed -- or perhaps more appropriately, we went off a scalability cliff somewhere. In the case of WorldMaps, the cliff analogy is certain more appropriate given the sudden change. This is why it's so important to have ways to measure performance. In fact, I'm not really interested in the specific numbers in as much as I'm interested in the trends.

I sat back over the weekend, dove into the logs a bit more in depth, and tweaked away. I deployed some changes in very late January, and looking at the February stats thus far, it seems to have made a dramatic increase in performance across the board: RSS went from about 45ms to around 5ms, home page from around 75ms to 45ms, average page and WorldMaps from around 225ms to roughly 50ms. Still too early to call it complete, but without a doubt, a step in the right direction. (Despite a small WorldMap bug a few folks alerted me to! Sorry 'bout that!)

The most obvious (and cost-effective) ways to increase performance would be to boost the CPU, RAM, or network capacity of the server. None of these were options for me.

The first thing I did is examine _where_ the application was running slow -- not just pages, but specific method calls. As Jeff Prosise points out in one of his MSDN articles, once I/O requests start queueing, application performance tanks dramatically. This is what was happening to WorldMaps.

My data caching strategy was pretty good, but I needed to open things up with some asynchronous processing. I implemented this on a few of the pages, including WorldMaps, and I believe that explains the performance increase. Be sure to check out Jeff's article on implementing asynchronous pages in your application. Although I implemented my async tasks a bit differently this time around, I have used precisely the methods Jeff points out in the article in past projects -- the nice thing is, it's very easy to change the behavior for debugging purposes (you could even do this via precompiler directives).

Finally, especially with WorldMaps and to a lesser extent RSS, I needed some better client-side caching. The first series I'd like to do will focus on how to pull that off effectively. Specifically, I'll illustrate to how to do this within HTTP Handlers or within custom code. Stay tuned!

posted 02/07/2008 02:34

avg rating: none

comments: 0

trackbacks: 0

www.structuretoobig.com > home > blog > read